From 4fe16cf61ff4738e46a4b680c0c2ec05cd23fd3c Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 14:30:26 -0700 Subject: [PATCH 1/7] Add generated skill blog post Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 42 ++++++++++++++++++ .../img/blog/skill-doesnt-work-yet-cover.png | Bin 0 -> 28963 bytes utils/fsPosts.json | 1 + 3 files changed, 43 insertions(+) create mode 100644 fsPosts/skill-doesnt-work-yet.md create mode 100644 public/img/blog/skill-doesnt-work-yet-cover.png diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md new file mode 100644 index 0000000..ae6f993 --- /dev/null +++ b/fsPosts/skill-doesnt-work-yet.md @@ -0,0 +1,42 @@ +--- +post_title: "You Generated the Skill. It Doesn't Work Yet." +post_author: "Sebastian Huckleberry" +post_author_avatar: "sebastian-(tiedtke)-huckleberry.jpg" +post_author_url: "https://twitter.com/sourishkrout" +date: "2026-07-21" +post_image: "skill-doesnt-work-yet-cover.png" +post_excerpt: "A generated skill can look right while overfitting the moment that produced it. The fix is a tighter eval loop where the skill and the judge make each other sharper." +post_slug: "skill-doesnt-work-yet" +tags: ["developer tools", "ai", "automation", "skills"] +category: "runme" +post_date_in_url: false +post_og_image: "hero" +posts_related: [] +id: "cmgeneratedskilldoesntwork" +stage: "PUBLISHED" +archive: false +canonical_url: "" +runme_blog: true +--- + +You generated a skill. Good. That is the easy part. + +LLMs are not generating bad skills because they are careless, or because they misunderstand what a skill is supposed to be. They are doing exactly what they were trained to do: infer the most likely behavior from the context in front of them. Given one repo, one task, and one successful outcome, they will often produce a skill that recreates that outcome beautifully. + +That is also the problem. + +A generated skill can look correct while being too specific to the moment that produced it. It may work once, in one repo, under one set of assumptions. + +## When Agents Generate Context Blind + +What makes it sting is that the eval has the same problem: it can certify the very overfit it was supposed to catch. If the same model generated both the skill and the test, the eval is not yet an impartial judge. That is like asking a team to referee its own game. + +A good eval should push against the skill: test what it is likely to fake, skip, or overfit. The skill should push back by revealing weak rubrics, missing edge cases, and shortcuts the judge accidentally rewards. That tension is the loop. Each side makes the other sharper. + +Let's make that concrete. + +## The Setup + +Say we generated a dependency-updater skill. + +... diff --git a/public/img/blog/skill-doesnt-work-yet-cover.png b/public/img/blog/skill-doesnt-work-yet-cover.png new file mode 100644 index 0000000000000000000000000000000000000000..9578346d4b4ed922d9ae85cacc35301ca6a77dbf GIT binary patch literal 28963 zcmeFZ`9GEI`#!3KN~Dk}QA#4oJQh+?nKDmBrp%Hd86znRl}a*3<|&y8Nud%l%dAxi z5ki^m<9?p+`~Cgy{d(=!{sZ=BJwMbVYpwgfuIoI{<2cUayhAkAPt$E>-%3G2L3c*w zq!tCmrWX_x8$C8}#7|!Do1n!X+!iWYY7`V+yc85Sf+#4K@S_`}6cp}9DJaIxC@5r} zP*AX3iFxLICu zvy!p6YK1>2M2?C`2p*Lb6cN@D5tb2^lo6H|IC@m(=uyqthgbh^Z*X+Bw6(te|Ne%( z&jp(C2EPA(gPX0Rm8;t&$1DHu@A0VEx&!YKCEnw3$=u3;!qC~B`?Qsl)m2=aCAXW6 z6}Ps%t%Jjnt!Lc0PdZzJv}cEv+o_>7853oem@;+uutDBVAQW2FvYApWxd z6!#|L&oetVZ@?c4TANM~|9hY5fBu82qP5jsy=9 z)xnZ$YN*;J2ON#PbMKyM*_BJGs$u3< ze}4^}d&HL%6gj zWQ*T&r@m=tw*IGcPbi;th2oTRIY?rHe??%q8vdB5?F%-J8Ga3AaQ zKf5y@M?ZPO#IKWa;?`DrdV2iCxg}9?u=09)`S|eguzudf)cL;M(pTL^TWVS=PS*@n zTzmWG%_b^3b}>GavMa$yHay^Aux3n4OPiUQaUE$qpRzs`-d ze~Ool>T;g_{xPaG;L7}Xr^UOw%uFjY@3?Y}OLsBhzE8SrAxlNA9Z*`u zl`EQuWj&{d11M?Qs$ZsSJY8B^;(shHCFQfe`Zst`vZc1Z{<%qcfz!ZCCnu+$UAc@Y ziUEJ8hNgOp?T23nL<;DyjioXM)URJoqjZYnQVR8B*rRig4`*Ji zTKb%Cp^+e$$xD}p>rD4wTTJ&~Un{ojT$=qPPq&&taec?;!tJiEuKxc1;-AR8>T#?b z9I?%vY6)`E{QPH4EbQ#q85qo*ns3@rE-z12|Lo3__gh&oE_E8S3UD5*+_HId#`)xW za$P8M-!@9+pS2;3O|g<((pSIXaL`o=s!-9qm%31XHT6{Ju7|v8q&s(7K5pca^E~P{ zs);?qy0R@bZ4lYVU@ahS-?dxJ=DvB)>hj;!m6f)_XN(8c?<~jdXP5K563#BZtx7-T z&uDA%)T`y^r|<1SIJ7Un;o;#qsFBc*JzWdp`FMS9^nA77YHz8t&@TsSr7sk0x%jYB zn^yAg?;pKPf6sh<$06x-_!lh=HxI++LJnu=sb{A``5!MXE)G=to0jS#sJy+r_#ZP1 z=&AOHQnF1F_Y))DevbB1Kzq7|q+{9DBF1 zi;aGGcrbWyS4tqI^qYfv+9!GnXTN<&HW(Qj z%XEBuk4@B?@2+69r1M4bq#bOQxc}qFk4IMzVs-E_mUB_>ZaCqKT}B$&l$@#BYBz{n ztzP}p7tSFm;^@A2@80V5wTh%6F>cym|Hg!g$w{2xA3J!I?|l3F#YwOe3r=s z!lCQ=g=_-}-=(>YR9jiSxK#M1q&#cy?CAB&j}g#+e)W3;v(QDkYrpy*KYkn$5#c=a zs>)N79q^m>B)`O(*p%v$yxM zN%__2=;$iM5ON&S+A;4x6MrWAbW09Ii(XbQbsBgwyX*a?O>Z1NzI|(fo5ew8j#^w= z@<1{+$TzopEPd%s(5I*Jq4w;OPW{QJw<}e9pQ*xzAYG0dlO7+r&^s_tb)muocckwA z=7FU1-~_f~b!k3bJzlF^RbIY2;h6WXLt3>hvXm5wTr~8%HRCQl(@Z*{ou=w??V5&k zoe$D3zS%fUl?@U0)Njca`AR!oUDv^J4mki<>^9a`*D`fcacyNWTPJfL-}`vk(xlQI znhM;F&ASd8v}By;aY=4@@`P1LNJyMcw8XJ**Up{$p63@7Xe}PtClXj%TH0WkboLS7 zOuL4B`OBB2gU=gdCG&0*uhV8`iRtO-F*LylkXNr>x&CMtp947_eTVC=r9zPR7g(Lc zB{H*EnVXZpd^wLBLj>`w3zgfp^M2hP#EAv}oS&Rjx>1E?iNX3Z3Kg5YU!{4^V3qg! z;twttH8;1pj!bQ47I7DHtybz;wk6+l<1I-_u@a72+C5064-abi%{EDr#h&PVd@OkL z=FM11=jUmjZ-eQu?c#H5xSHE#5qECgs*4e~??|Z;YKRmlb?6bPD?sA5>Sto1c*V_N z?fRcIJv%qIvM`gJXoeGk%C}3Xwzbt|v?VFn@7;s5Dk@LJY#*3MhlG5HlXms-^>rVO zj6fMyi@r!-m*x2~#)2IG8<@doZ)5A4L!vc?+@P#M^GI@24 zjSh>yzoU?ACKHmDqQ8Y-;D{nE3GspLD>BeM() z3}~-D_|(^DoZnQB7|znqdp@gt948u!sgL60H9vlnX8S%Kp7rG^`EX?mN5^=a$DIex zJ`yzU`tTw1(%WED|JCB8zZt??H*eO=8AIk7uDRJ>zR%dW4`;C0dFY~v2Y>>Ol7N7~ zK-Yu&_pkiwFGqF?#_RTNdH=N);FB#|wn#`wggm{CEZv=J+}+dDGdOth$KRdI%=2rj z%SC>%h%J<*NAlaZZA7f1nh;e}*Wy)>SU0-~3#my3_UC+Q4zNY7d|N9^?c`|E<4 z*>JwayXOdIym@_Xc?uOo+I3`Q>1Q4?(gc~@hRE_+{LSVi&z6jRUS0h=gp7w$t|4~? z74$vMai&)4Zca`Vt^Ok0b^wXdYpK-huQQT7DHA6>J50swI^K76{lOXr#xzC`N{EPP zH8F)^F%Jn%2@4B{Tsv%=Us!07qr)oANbU8w2DdQYotNjS>+bHZCY@aibjV!c#v*ia z1%#t1 zA0=`9_qV!G7TT?-RJzYiqz5E)Q-K-99D8+4v|pIjAoXBF$E?El?Ae14*I0Btckbg> zW_}%P)Y(Y>A>Y3jaMp@0{Kl5(Gdd`pul)z%|=r~aB`a{^Fel%6y(fZeYqhh-)P7|-QsM~)nc5;E=b zIJn>S&qPmK>bdO9#zvmVEQFqr!3#6M4|3nNbLY-UN|tV6<$4T>?zPnq-En~1cuZaZU zk!m37F=61Fsu-@^j##TzFi2F`KtpqzUpH%i@N2~L&wdA~DHR^fNRs;UlrqOI|lWfVcoHD7);8I*HQ3A>r^D(MN#Gdt7^9QqnY#LVC&YYTi6fFer>J#U^V3fgd-v@lexIad!|S!WRQ&Ym(0FI| z<))a_g}*wlhq+iCnfWwdH#G3`^77^e1_cEXM617CbgXn6KT?a3=+L>dJ9g{$Fh#zi*y$gy|_im5B!E5E){ zSW)5qy&=+F{U+9!fH+Z6QA@HZDNg-mt(n^C$VQ}h%56<3w=oiqSt4|PJ6Kr$eiW)M ztEiy;^{jwQ&jtW4MaO&9%}rID&Iuo2_3^PxdW{eWq~GHADC8@Y2i0n4O=D_#EzmHh zwv>?J0LQH3;9dZ;1aQ4RThG1R#2P1ez2JPZGJpFp3!o3No5(Qk_GVVo+r#KpRaN=* zpC5jXf~bhI_gK(4t&jPW=5e2e7BD6xiO`P@j*ckF`w|=&-nVTI+8KS!+ap2ldaXFV z^x{i5TXHqezI{xWAAWo)zaGXW+TpC^<3r>y6t6{qoVn^`R8U;*bA$YSN`GS`Du6?P zDQm^!Wv_KP(@?v*&h8R48kM`RAc9I~i`(!ta240fyEu?wrHA$^Dyt))rDg= zJ$+@M%Ny9CE7!Q%b7tSyYmu!;qxWKCV;e$@wT)i+l+zdHvae7k7pAD)Zu|XfKvqa7 z4{!AH_6{`;U=IP6n4j$Xi#$vgLDHn-kmz`Rp|7Qn){=W;(#e0(@a4J!Q|t}kIeU2h zyLYE&1R8<4Gp3waI5_$c--wT&GinDU?B-`?K4er`&vnQ-oZn4Zf8&VhtCc3nAwG4D zoHi6x*=xU6Z~;|^CrgL?+83o?v*0H>K0&buJl?0Ci`sPI$U_lvakZWqsr~zFkte3A zx^TkcbFwd|2`@yv1cZhD&>p=Nr)Cu&Yx&JjL^&WJ0Q^hi$2CR(Qt*@*#JRs&+t)tl zYF)!cgcd#+^rA>PoIN2S0l$6z{Q2m@h+g(%UPa{njUyu?sH_d&Bwi|v@Gvygy?y(( zr^nEQk(pT>JJ!;oIrCnCJvtHw#Dl8Rib=*`Iwplj|=VJzn|yvVKeMB8{62IFG*2yiJxn^mkps0G;A+?{o~Wq z&c?Nv_rqLI+_C`s+b(%_lo3#jSpfA#FH0vACt`VFVG}j=c)!NUDX1{Tm)=slglA@= zVw^)xvL6kn64@P~DMP#Ku%^SYG8`}D8q#{Bfw3{lkHV@KsB?UhYkSjEd3m1t8c{!z z>3QT+#UEo_B05+Jz)YmUoWBQXsHhf~m(yMM0)gb2(NpKj`7BK1mJNMxXRy%3|n^SCh{YX=+sMD82Y;Plw9%7mWPbCSV66=X-W(oV4wK+FDI`pz>VJ8dAHIOxb z|8+_{)|_UMqdHR6Eec$)b~ z#`N#sTJi~>5eZnQyn@fYy)Vt{NFl53U%4cS%rP~5f`Ia*=tTwM=oHk)85e?0WCt3 zCB!PITA*&naQ8SRNLJ@(*ASJVD%AP zc@}~SG`s3+Qx|P)Mv*_63P*8)qD#I95bo{(-?*0So{l%SXe-LgftV(;^IjFac;N}0 zfygX%?Bo5Str&pN_P_=ME8e1I0sEQ-aD%c`;Wck3UL}A#$d>B5QzL)W3z;(<*i6Wz z?C3GtkbC#G(-N=Q+1bLvLi?VA?~+44?Es#QacNUi)$4mCo#M=UKFP25gZ!xW@xOkt zIY@+E*t`}AkSUgZt4_9_m&B|aS;cvVrDbGr7bV~rXAb7yxqFv>+qO8s zD>6$yU?Z+-`_`=@BiFO%1%N1B?!V5fUh8d%jV5Je@YT`VF3$egzIn4sm&v=<)_f$^ z?&lXcfouQm!BT4*0SF15UO5z z=6On7T$~f@vM}@Y;YD?D!=Doqawv}2FjaupAFU7O_T8kvuZK660YCuX99X@C%>{ov z4w0fY`LuvXywsK9*FjqtH&NA32ed{3fk2+Q*ta*T9r?S0fX!H}20(YL3e7gw`>n2j z?8IVPT3Uc>50$L3LOz2iE9zxI@l}!gQN%xt&mU>7h&^qE7CJ11xcNeT$ImYmg+E5e z#HfSSV($(W#uXR41E>H6pp=+%adEjQbs>il$|X`kbg3lW*`~rw%c}CQT43F}n@YR* zb!>2?Y}J-@z#@E?|CCo$Je9jHl6$}btBYVJwMPknz~p)S>NiYNjn?tEcXqBufFT=g zNptJA5d(2YC^a4?m;uwba8%dS7)dRWg$OTzYY_W7 z4qGRuR}dzOj4B`{uYiMKO^i@d=su?lU{Ab&=K3qHIU6lY1A&IJ2;Id6l)I*z?ZrAT z&y77atLH>*nS1}{jX6O4g_mxvQ$ZT2PyqF&DDgc>O1A-lAh5{XELm@kmj#m4cW@Q6 ze9z$P&P`LWbz@SBU005@O-mv$&)9S?P7{J8>+oGj@I^KeW%c-AAU1NuFK}_?ySoij zR3h*@A=Q1c^8SAJ_8i+E?P%Dfst1wJNN^TGk6lq)FG0#l6@O`qj12C*DGvwm0?NMU z)?twM=NGF$K&5kIb5VEPe|CjbSL#0kzy_2|kECVnd0w10E;%+9zAbHIu15o}?cl*_ZFC47OvU7ukngjf4#L68aZ0u&n<1L!w3P*IB8V=Mj|780Y7;W!kA zSmL+l5O;8JXj^iNm%IKO=OwWF<;#~@Y$Fx8Ux5Bq4f)kadNtVA_UjMDpe@(2MT@hC z$Hw-hoOxLLFoo9XWI+o(p@$=g+m{?0oVgmCnvg>n*E!kPt^*zjtP@9{`LMwYrQ7s& zUB=1EbS{b`sxO?jogRN3KuNwOD=kg8N6w(B&$cVa5a$v({qYys=7qV59_=Q%uQ(dn z(1HQ6H;E%|Gz1vf-_j^gL3f=0@znnakOAJt@4b)>SSEY9ISw}k)+-V}8Ss#F_wKK* z+}JYC+sesf9hncStNq)aZ@97Fa?I}5Bc#W=5B$2|WJYv`H5BbR}5Eb2_oi3}PZbOJ}LGDJ^Yik)JotNS!Rj25oKaNMj&*twL)x zmvHW?%aKI>Rlc_;*OhAk!e%G1H`q3kiab&r_Gmk&Ov>-Ub2D8>^Jy%UMHeDSB$AfT zjl53bw(;Mw@fqvA+2E?=jWyr6AWB#=Xlu8l8m&PAuJW31h~T+B7{#@FHvuw7e;OxX zRRko2TWrL@PE8O4auy5}XJ*Qyj1cJk{rw%?e@9!DvZL>qTUgu-46GGLAasl`**iHs z|NL-2AD_yUsWJOO@g_y8@c3<~!U%Ur%P{e4hEglw_BepOsI&Zw^H2q3vh zg-0&;RmvWY08I~c1a@t6i_i(!NX2w8TV(`{gxpFM%5>=2+edlvGVVL&kK73lKL+-$ zJaNP#dBm^x1?84BCbXv#_>&=q1nywhLO4DiF2xaPy{?U=$$CYUB9gOQ_SF3}wOH%g zc8Y-a@83hlMLn$q5F(%TRbxM7w(ay;ml!q{78XZYQ(I>4!rOq?*+P0NjL~}M=qwgq zTL$hsMe@9U9hs3Jy?B37X=!)&10^M;bhX&W&9^H3R=top$uT$zjv$Ou2PddRXmC@| z6!Te=I)R85y?nEWcD)~FBQzJYDXq92L2odiX;p97Eem< zQ#Xf})qP#N^HWh~tIfp3L@kvq-Tl~G9(T-7AFuKRAVlo*qJ0CRk6l5rT86ZwxqF)^ zBfA$+sFuSngIptV$Dt?(*HSy3qK#^2^Y_*R<7lmx1Cfe^M@JiiJ_5czH?7{o#Pkhn z{lrAvS`QWQUzR6NgddcC(mjTNiEu@$#F^=7gF@?YDMdh$(W9*KWCunlv|Wy|i2JI` zpY|vQkX0mll+c7lg0KCgoN(ONZnSB|U(Rz{!Q=`gJZN_rro#X~RMl#q4k~7|$v#jE zjM6d+I7&Nte;Ygf8rTJrK6bYse-pY`nbD`u8qKgd{ENyl|230J)!b+5`^4Y!T8SFP z_mVPsbNzY&AwKwoaAqNKOGEXp?UF&F2SXW!met|rrNO&*@1^=MHsFuu(WV3$cRTDH zv(Q%sk5k-1%E_%vLKjVTZ+8D_`TN@kMwfu*x3vO;b15>feU*<++!sU{<~#qa7L0e3 zI0VF_!@JT_ST*x5RHS`AqF#jt%;(Pr1FtgldSYZEnul&(PU8HHTKUsn^jUH;A`t!4 zIfV1L)zL83%|Z27QkZC(y2q7bRi>({!%T6)J6Hrt##Mh-?7`_`eT@F~O=|RWZH}VD z@#M)9B)B~?ZhQ#b?Ay_bii+4>0c62or*Ztt%F0S*W#zVtcN9uVANWM>n@tp-l|l@%9K>EibDy)k6D)-01fk+T0#Zz19-Kc#wJdl<$80){RfDx<84U~#!5S?ATA;Bae{;xs z6w7@~Oru!W6kUzU0ueboJl@a{u(jxBcg zADjd>C{U|>EN#KV_Linz3W9w@U*9q+GbBJw*Y8Z*yRw8Ja$$ia=3Z^sc2nWbwnr`& z9*2(qKC7`ToNE-oF2~T7seL+s1Q2qZK<`v}ExpobMf!1+i-X zw~e3{BX&qH=Si8n8}IJJ7E)TIp@Ok1rwGCBd@=tgiLft}5B<1~`*;E@{u5cA5exTz zoOu!*eWA)r{Mg59*RE~fwhfAOcH#)=8#J;H2pQ!mg>Yq2uzKJF{o4m86&}|g+{30a z^XUsfw7YSEId_ML)Kp55*J#w@(0g9AH!h%|Hxm_uz8)PNU9Mqa+9gTxh~~*4ck9mq z{!CVpoaT;>xqah1f8^>#Qxg|LNUQ)4NMoQlre4&qyynlYyipxhvOzkYF_z$Dr&Cr| zw*I~RAc&yQ1Q*oJ(%I47|H`+*p{HQX@Eks&Hb67EwbFN4Pbh%sjzH{Z3hRdaIFH&y z7$jDIZY*HfxK~7ThFn)ig_P>tkr6?EPH%U#S~lCa2d!COZ;) zrsSNdoJ{8Z(AP&u&cTC?jSqD_?l=uR-??)qN zYig2k?gI#vQq9XMtxKxuf_WPtkq2b4l7?>{+C=jiF^m{}g7v-zW`%dU*pVzd^f zh38T$AAsnP_v}UmJ8g9OTVHw3ylLrsl3CC|-7f^L{bI=7<2M>OZnxa~br$dvKc?aZ-Dq@7PxS%=2r8E+xOk zlRw9Bu8>Ui4s(dxJrb7tlw(w!wZJfHzb7zfU-ksUK(8fw(@)8S?(FI5$#?f^rj*ka z-c2Wn^W5GDK~$^Hir=rAdVm#+2^1|T>%ZnBBYy%)uTIXH%igU3kG#ID%V?^GFpdCb z2wiyjC_#LqA#DHy+xrh6&iFK;y*&n5viJhgbVhfIhYwAndx?EF$t>aITL+15*_}I^ zkNpykrvq)s9|1(Cq~50KP?`RqIhh4Y3yF*Ksf_#Cti4Bw#dWmaIO947dR+z#wKPDz}48BfI z#wSn~LH%G9iM%oxt=lFfefG@6^-go_u_g zKlmK%Kx37gW~L=j-p7qg5p;HF$NKsCp&MQX#UbJz7f&Dk^`*H?sW=Gen%|!>8K2-! z2qD*IT;)4-lM8hCtnYlMeqbOJcF?EHe(G@LLTEd=8!hWwKW-3_0Gv%R1+-u~&3fE@ zyaS9rtM4~_H03xkF5$_^$;}TbAh}+~b{z6wg9m}6WUZodha9}2hx%p%x5c5m*RKAV zFoZ}3%^=%^AbU*d`^Vu987?IWYIJB^FS`2u?CP)Qb5EpQt*=~pL8g37H0DQJE+L&j zmFc`B2(lk5Vns5$BOz{7>LmEfVqH~jd`b`W`P+Xzj zX%`kpD+tlN|4basct~NrC63urTQj1f^wIQ|bstaZd;|3r^eoTww}efL!jDM}`={TU zioQF|E1=C`_Gsrv7h&wh(98=VHt# zXelnm6w00Xl$to9LnJB$pj(%wdj-BQh=BqUyelheB_u!d`!`${ zCQGH#N@psRbo>@~?%DGS$j>F5tiP8W5H(ospJ#n8?2C%~Slg-Ho6%u_5_}uVoi*AsoGh7D#!7Jzt{&1U#+VBi?fp#Ib2 zzJ9Co`ik4@-`(R{US5X9$hx*Sya@_o-Gx`Mu}usQn~gS|R2Kdt)EvY`NPJ*QD{5a3BL}$M?-_?fc+V3KP_a7x&wvAZL(L_WcKZL`Jh|3LhMmL>IApA zetX}VuEAC1;2PZUTs8#^gd7M$#&wR~FA!uhrIdWp1OnlQ(o%vhs3ni)>C>mtv8h0ZKz?=Zp}8bshAMXG(K68nuc({Z z37#N>mXVG=coPc)!|=$6_4J`JZTJU%)Q590|NX;)b{>+D9n=X3w{Rqi!hV2#^Y$hR z)=L5@a7Y|J|11bn0~F^3dH?4_YH(UrFBpG=H=*L?OQ*gP!Q4PFjpl@7#+AF!0K|J& zovu@Vcr3dg4RrFm<1Q{PO2ON96j4K)PRh>Cj^S+}Y20V}6uMU_*k??BH#0Lc6Vn_v zD5a7`Y#{<|7>JlbWO~D5yU6Tp`z)RVxV-kxPIs&$iS~o)k_=Kbx|LPaGc)H6um5P~ zL|X|$wuD`W4J=VGV6FZ0LKtz$dJt+L%z{0+#**_&YzH{DD_0v8*|Z|46ctrf8^3*c zDC$H>!K3)_yk!~4AKDgBmXz~<0|_I*^GiyeLtjLEiL*ETyLD+-&0ho43P|vE+K9o6 z02rfNSr-RI(9FyX+YuiZ2b(JcBcm%w6@+z2n5|E$4+Sz4YZ4BGLytK5{?%A_v0RPiaT9Gp$6`NoVfkO}~D%p~ezY_Qj4KG^wR~DN>6Ll}{tLys? z0FG8*mF0&Au%m*wobz!Q#110Q9uO1_=IMm24TdLhTv*nHd=l;|`_8ORbgFBw&0w`4 z42=C%?Pl@iuTcPU|_epDHx=%nUOzT+(pb~Xm(&7;nEd;_nSg&#l4G_BoX~gWIl5fH% zpxs;qb%jJbC;m2w9*b3A)4I>43A%-Z^B}0Ug-Z|gPjqNgTW#P4Mvm|Jv=eT$O+3}w zk1pLVa_rOFy;+=H4XQ0r(obYPH0&V-Q~}h1oMR{S9C%|our#$)Xxr(TsF;}a<69rB z?eBbhX`08%)>ayLrmo~p2xK^LVCi}9HqALtj>d?_i5u) z9B6)Z=J-Y2>TgJFy}iBGYgsTjfB*S&87G6(x7ALJ-QNBub}F0evdEDm>Z_bu=&JiaKS$rTLIgFi;Yk#=!cBm%C+ z85nXu>$?-ltzx!qCvRsqwl#2oBKxf^UY;Kc+)@OKn%cv)u(TmcONW107Tk6{Xo2(2 zDK$s%yhEj^ty-RY4?ETj(-U#~#}SR6w^(ih8M%jx1=?hO*WGm%LMFt`b^t_2Qz`4l zCurxUUhae~2%mI&$MHQYZ(3W0VaEjPi@s7ZOeXJSWi4|WkeSKpIW+%kz#GZK%Ymc~ zScdbI6&=(hWmU}yYv9blwV_nky*L|k9urT@PP~AG2y5}l7eNq#Got%=*%Sxi>kyHY zgly?6!}SZ|^&Em(J1%h_%TH{8)?(}RUYyyj5{YZ zr?I!l3f3dM8=^xL?xK(}1yD+wgu;&gu)=W{^0e>khe_3p|xdqYJB`j6zYk%c)A3R7+{f<PVwBE*|%=Y11ZxFIb(0G?EE}A<7-b6ukcaieEc@ z5pbK3#yzLau!GD|kMaBHDM2Od{rK^m-0cFFEYNAx9-<=$+^Vmy&*{XQlbZ{+`0e|5 zcVJ;u2FN1HO`d!Cf$?D96crPjBha(tz?n2;X7u`{e3#rHPKjRDu)d3Bf{a7hzrMZS z5jk{LCrjtU(9k+cjG$4`!tb|Od-UmmxwLmLB%!1LTtW!ZXHadJ4D!NXQHN0<0sjm>8FKy?~oB4pJTz3*_>i zL`BNCuaXfXq^V6r{_jjl=U0$LVsX;-chdzTqCj3Y^HAvDh5t{n=RECGE z0i9rS<@_t3>jw`WM4^jhJU2>!D43RHktLAg(P&&-TlK~q0#YBUjXq!+4j5#iGxM?o zAjq3}J>Nf-uXees%L#WT;4ws76w67t_d=YdMgDp1-CF}oXIGa5E*KtFJ}BM5Y}tSx z__Qz^o<~|}S|RLCoH#*H0@QUt?~}^Pyv?Vhh=Nt_$^trtXhxUW{_~3@y=yv#Iud<* zA?eD(B&VYC9tYACr5o7-(kVQy=zhRq(hLm?!D8#Kg*+c=R{sJ%O>ENXCW%UzevT5a zX~ZH5gwO0qI2yjWxsF(=bCLYthle%MPy++9TV{PXe41ZE;?Gn6HNyJ|g8-mSl;_^! z!a@OPMi_uFDsdQ_m^hqpa6rYf|O%e(;rK=o>h|Fg}U_Eff$h z=%IhRP5mzb*TKH40|0`#ZzD97YAzT$d5` zd-+}z7IKJK-l0D?_^JXpUj05*xkf&C*zB8Y@QEiUS@W5kJpfq{!l=m_=`&W@nWLmDhMa>5nvI9Nff z#kcKZLL*W%KHw-^oNwOjYab*Ge{L1sOz#o2|GJ2Qy9ufcORlQ7?EMzP4tF_KIhjBU z%>hS~H9kNsLvA>&X=ZKRg;qPh)GM5gkZmgrBY&d*8zP`KG2uF@FL@7Uf@MeH^KJ{O8YX zgOTCk#J<9xjgK7Hu28a(6gFpECD(=^sPFBT{tZ%!F&Ez@090gm!kvKT6bc=?mn%iF z$mAopt5>yrO0K{G<;dcVygXkg6BkR0)hWMfB`YgiHL=f9?Eib3%Bh2{so*Diueu}s9&4axyJ=jrR#8{cb2`?bm|{KZlE#_338o!8T%b44(^kSHSmOA($W9zpcL16S$$nJf$#*E1 z&9Rai`G+MWx=?LPb#vfbz)2xlp(J--VoD9;o#k@kj^7-la1~+)gebzvT9h=3y!-3d zFIcxE@n!7cJK`Ro{_9S>O6^oSk=q-}+XmuhPV)wKbS6Gcg?km7qmxlDK&?ntP?*qE z=Qu$(3UVs5Pa_Qhga!j3RU98B6M5>TUY z289ehoKX9s-V+!%>~BA_fvOl z4)_L%ep}TR<-L4-Hpn@_gIg`H+XX8#b+f)bl=;pn2yy{ReYXcCMxbEQu!yV+<9is6 z8He?i*-s=ZWEncJ+3L>#C`4l{-vQ90e$kq3$oiQPF7J&YoOZKiNHq=A7Pfs2y9cejQX%cL0mz%|82|E zo*Qepp^*a2ErVuv)7f=PY3Nz4M1kh$(qT6ea z+8$Gu2;2*kXYimZ6-i_BS2DTzseGRGd6^;>wGKyYj;|4O>h}cDz)`pR^#-TiAGB7< zS-ju{&fbs4Xl+f+OsxY-&dCLVRJ5=b8JQ#((6(bW-TSB`12kPkSxv<___Wg+(M4hY zNa&j|>D{qMe-&svt1JUe%*W^fwoGv-1S47=j1N@&g+|+9c|x2rp3;^;q3AIrCPH`! z9#qzyYYU%7HPK$=7EZO z>02k@JB-4o@(-gWyK1vb;VfA%*^s;?N%<5v&Oj0Q#2NsbqJ|*S*v!YyM&*o!p!*6> zjy;Kug=8r}-^&EC6oC&v2_m(|nGwz=%uPne#&RriQoiSl8@$nhbcs@+?h^Y9OpTbD zfo3~q(v5=;?&OAJ0d#oWbD9u$a3AQzpxp@MBzXSWNkCB4CulSoo$n=kF;*o7LogMv1}^Hkf!2e<>PLi|5N$@3NC*coW7ODJA|ij@%ZF_efNAFOt)EG(A^ z_0u*QW1^T0WCC=CsDT)FY)K4g?qk3MVxF840n^TG7U0~o#}C6*P@Qn;s{ngM-VhgW zf}9bu^T6)iyHUH1i|r0Ltw-}iS*&p`Y+yW57hnL@Z*8=dXNE$$rluxvsKjYt3E~*E zTaD~t>^vr1;>`#P9H7~)#6+wi(LKF>&~n6Z1;!&q&yvzoNnzo|-$TTlnsMnb5HkdE zcz8I0{V);(_=>{aUglx}Jm==5dKP^Tj7X5IAg;;8iST#k7`npp178c^-0FZIbx=)XJ>*a?= z&_gkjgE3Aq03;tUp{R#C1!q?dcR&Ha=pt%G zX7H|f=oL=5OO!SJ_Fv77jX7xM0HFSa^|ozUofH0Pc%MN>iAHvB^EN%`GDtwzJw1`> z?dIR$P;5z~Mf;-xvf6_+OH8zbTOM~I2YKe>%wR+IU}8hBDg-3pahLWX_?%Eh4Gj$T zuO_u7Jis6r>~PFXM28NDE^s4wN6q^xd;~DcZ^UjpKc7+2D0Z0Y;}Fm{3|>@JI8cm- zpR9Sm6lUHpLpDpMd+BA8Y?{j&>2#3tL|*|M(a+ zg`OPQk#_rb=Yf~Sn$+^2>VQ~_UMlcS*0t;%|;lIkn&OSai z1}Rt<&QX^smPlw>QG&*x=$Obj9QpMH(kaHu%djA5CNDsZaKAW@eUF8#f?cAdp|^>p zhVF+U*!1+vkS;n@ccjIl>N~$$Ov7%Hh%Uj&)2E%mh)@%dlC^fHtc4=+8ygy?E@fkn z_Mf|1ubSVPQC2g_^^a$(* zYK7@arYXtXlP6F5EKin_Y+IU}iE($f7+h3%ZfdL{5Ep!14H_{;Wlf-Y+bGF$<|96DO)I3y#6UYrvG#xI@q@<*=>(`K!{vC?Zx3o;5 z7pivos*F5bf0r}+t=c3a7D70oMa?->D<@yaX+<1@QRQ4slbr|VClqlx|Rcke>L?f%Fv|If6GWSyt2y?q5drx=;CtgTQ_`w#v=F4yqd0WGl(s#vWMhI^fR zi!${%EQmmk$0H?dFcA0LAo+MFrqp->cv1V=)aq1Tu9mfo1c z6BS{#3zeSduiU6*RNsG`qbNV$QJHujJJ*963Yz_ARj~Qb5oc$LVl2ANIKM9cqf_SdP+Z)}+ z6l03qn-UM{WZWk^K5H0chq{SF3#Ge!kEBq(?ZbQb?v)5tBBKFyI2E`AM_WH-hzJsw$>|n4odD^IMrYT{V>&rj0en z+1a`kV@;lr5On+_@89QkqWznipHbGCG3@dSECDcMQ=YE$tBmvUb+*J5e`1t_79LT2=Q%O#|&e+F27B>%4g7ER%k7-=}V zdb1!6XGjngME%E!ySt^Ir7hoQC*1l;5oeP0e4819G49s>XPn`;}hfs+3c4Iq`kLoe9+}T+SZd88t zyH+6`I$5NTER3Im(<1>eNfq^0qB7Tbn5I(E0rZQGON0)D2N*h~b;+i*GMFZ%E;q zW!MPsD!celBC0*{AOYJ|9N@xnjx$Ea#`>5~L64a|e1xD@d8YcJ$!G;&Y~2U7o9XFg zlqtO?O?Lz!$YD120`@t~RTJ3=l^%2`_NYQEW3<;4Dq49>&L{=;B7~WkZ~XPfvLgT+8U)s4psszLPr>H0SFID)VPa9 zShRUsn;-)(=Sv=!&1(pwG+iGIHQ;JeSIRx_w4%}x6GW9W%I>><-inMQdW$kFzfm#n zaY(A?PvG?a0yg5U%q;qU&hzj?rdXP1pZ20v5CWyO{`}$+2P6*khe@XB zn6#ePd+}hBeAn|3WGFXod{8`rX~l{Dazd4--oEGX=Lh(0XdY?e(R~p-r#HPNVMy8r z6~&g02QvnPV2-?MF=0^0AoB%IuH(ecF>i)sYTX>ygGEVc_a}ULb#+#d{4vStXcSL= zr77B&tQWTU2+!+5=G!5cctiy25sK~AAZW0ZS7D{si zGV&|WV98{e&|)hzqsU~f0?Q9L?kKR}-PkD`F%aIqG}ns@|7%9%_<39#F(WcR@8Kq4 zUV8`8mTCLt>0>C9AqT6~>NUHKEnc0aFr*J)P&q#K2Aqb-^8%?nyBWjG{%L1xE zT|B!!(5P?fTMX+)Q7}_MY3UG5(>TVPW^V#aVeG3Kb~l&%O7mD#F-gg5C?%JN@zjL< z-u&*rzn!UJ;)GQaSmf*yql@Gl6gCvKgR*vG&O&xNm-F?IiX`ANO%Z&a{OlR-?~#=B zS+B=FxDsfl0>>A3Lxg)J^F8h1f2V6YHZ4B_!9*5qYHaMNuKn*C`l61Cv4_WesA+#G zK>ByqgZvw#MWuiZTN1XwV;BwnGd-a}9t*xmqM)cn~S(QAVM zRY%!~fe`3E0!Pl~?=bRsrX+d7#k&=0?11bBlvf@no~Wxit!~sOi1S3AoID0*Yq{!- zXU70rUxPLz?mC|G9Y+Bvg*lxwT=^@86Z0aj&OQ5AAZnW|MswHI%}vMDc^WW+&}fj= zVsM9iT`8SAuR;R-^5qM;+sn&~c!Y*sNeYB^fVy4XfO7J=)b2RyAXo{*#m}EVw^I%h zLS=jajQZyI%Zz<~Nmq=xH zhjDnH-=+VIQgDc(($6$yo23HqF;#1>Ui$$#j#*Bs3gTpzWGFe}1VFuRsnZ26uBRY1 z0!|9heefV2;MtBTnR$eP?ys$AS0m?=?3sRueN?yWelEDS@c`=ASlDK8;az{ zv|X~+I=gE#)hrh;p0|-u_BMgIukby`jP7yns9r!0M#HW8#fzgZhA8JilLK@PKq#&W4Qq{Csk7B6p{{5Gi>U?ftHyp=wlLb#?WN7cZiJ7?NyiDT3Cf&ay(< z*%`IW-4{i?UlbH%fkq~BgL2wlu0$QuZ%N9=@D18Ts*A30&(yf4!w^dx{*V6fzVrEs z2_^)zd~P<6gz+B+HncXW2&m2K4w7HyF=-?oK1uiNU$8s=(|c}ouODAL|4mcp4Wm>* z^~%Xd7}A!$&p;xnbtN+&(mHwDHU>tt{l9oHLF0{Xjir28XecpMJgkfUNNd&>!-Aya z56%=vT*RP-&Jkb}A4Dw*DbYT`TBfT;7of7L+ItgZX}%6J2@}hEQAyu@xi5F@!H~CR zbk~9~kR1zGa>~HwcIg9E#MYpvtfOXp%@HTTDF2-i+L%P&b#k^yRl@s^L!{C%JQ6^` zrGWG(?90E^k3x(+HNiPVt7^F%_Q?fZJd(sH+%Sp{V{4?p{fC?nK%zLLUeZfnyN~`v zro^%XgovuDDm{B*a2>k?7eWV03le&Fwm#YlE=Uo~+hU^pjG9Dw6AoTTZoPB+QKMhc zxH8|K2RhRR$}6?XzT8Avy2oUW?C3?NIm^b6xy_j!&uX9wW`|uzW^IfTdRp>PGMET@ zv)(PxLX~GBKGenNTu#;Bo&E{|`qEJg@$h$|hlfu^nz|l+;M$Ma={2-DI-GBM(ue5K ze1S5}3QP^y?KEYS+)7MtAa6m1D<~>*gD~mp`V+nGLGPWIK2-6|C-+*>1_6Lz?2TD$ z)9Qm3>>JOT!B*=hHoZlJ8i!ddNW=}Dsl~-)8gT%RY<(ojc+*%v@OBhFM{4((PL}Q4 z?;+vd(Jn<-86s@<+c)9=d$Gfb`2do5CcXHf1S`el)|6iPv*8eQ@6cjXpG+Jihm^yb z1awgA7+16tel89Ud+0$EKhSX@Y#8n9z8(l;c$4tJ7R;Y~M!Eyqu%9>3P|dcb)P8mP z14fMc&`UUa_alY`U<<_LeO-JGnuXk~HN}{Kf%s^9tm@vKJ7q{0AYRZ>{y>AJr?P}3-ncN) zspkt@nrEMP@jgGZY3|4VsLWJ5dKOB_oEG50VH6G!j>HKoXdpMp2$ zzBngGUCm+!?*6`LgGj zUW8~2-=bmvi?auEU)daFNK3IciH5uOk?8w_2k3)HR3Wr1G$-)5lsZGDu<^rPU*Cut zIXFz9O-ffm7wo$nU8xK)U2%b9JHZw(lDqc^Vc5vJIefWa%A067v81RT3Os&+?c`Aj z3!s0rnDo8=ehnZ(;ouWF7yP}dG42lw|)_RR00qfEeF)DVynb`Z?)uX=FN5Y3W})ZLq!S ztsC(GtbfLsTE8M-fB-OFE{CQzBFpXKFid`!X-XdYlhJY(Dhs|-h>ws)hfD1?UNY|i z2WVT$UHH}n^o~rAoNgKAxr*hOAT%QA`vPwf~(D) z4q~`JfG=Ts$^e7Tcg)dtnjWGzOmGW!$_)w*Ceb2Vq9zWKTwxk$5j_nib>YGVkccA} zD?QQCL!ZJK51+&6ew&9h5``Dr1!@`X*=a#R;JX}*<;9$Jabl9x%fTkbtOXF+>d}^X zGbHcZbR21|Hh8qs)c81m1{9V_Xwjlx-eGEw%LX@U-c;E9z8TM?$fpd$6Vi;Twc-*d zATG(ctq{|IP&LbVzbD^=*$NbPpvqgHYgv_e+5%yJM194u=Zq3h4EgfgFAKbNvm+ed6Ip6S^>d5b@sQ&wWEJr|FGJF}vC z75P2z`k}(q!`F~*b|K>8%IGe_rJE$F`4sJ1G--LBe*l;Rl5SthC9?)YyTOwYt4laa zfQJYl##V?%+TaD2@cg@h9}(MIgBathjg<#2q9Jo&pUW3mUrJCt0CWNf9jCzqKFB@aCsd6ZE}_wq#9*O#4h=IlSa`+xp&obbDRKezY${dxaP z*Xs9qZa;CrA@#7DqbHWc(<*$9id5tMy0Fg5?GepdaO@h5dwGqQ#|^LInK9jrk)tD; zQ$E;8X&`6LmbDh@bH3B*IlELU9X9ysyer~-n)owOXC7TF04IbG_mKfWVBf?)*% zCN*$Of`gf1g6XM4)`J&Fb=0;^?I{G#2CUvREw;hP@*Gk$r&Dw6YJNdhW*=x$K&Bz} zXoa+zOPjs6#HyzC)tsArePsVZQv&uLR=GBBI6zqw`e}4F&y9M2 zv*J)`pO^{EqjyV7U78d?h*}~2%k-_SL&POO0?aLB)kWxV?O|P zWzXqixxAAQjEG>Eqm!KS_)ow|W=L`i(pk3@Tg!q>$W6Py@sshkQ>Tx-D!*eX#&1YQ zG+U^QFNa7!+1U@jiDOB-?@=L>*9Qb};h4W|pD&hmNL`$<;klQN7F?7oQg{qG!L5Ja zVSN;gKwa%%$-Qq!E?piN_X1TVgyPKRr<5S(kGb!f{)wZ=h|DY8ef{xlTzH3%HIvZT z;#k#6MrLcf{D=8aTfD8u6%}a)yzLa_u6?9|nZ7Kmm%vr&=;)ki^Mv$*3&6IkmVN(} z9TOglU1{i-q3j2ljQ_JJHjJeaL!RJ5V?%e|yn)50)X=|mAtc>9p&m;Y>{n_4+M=(w zA+Xi(S%wzhpBG}6 z+5?Q14)lMMx{F=;^o!Zq9W5=6s#bcYcJTXmi2?*5r?B)awq?j`@gw3oK@dq#bPZyd z4fSND>uJpR;HOJ9RR(AMXBASIr!B8K;zHG>Ax32H_Rz-4N=Vt#-8IWs`u&Wy_ZbSb zm!;P}q5{@7Hd-Oe%p*a19mcBM`^^X%uwdeB3)fvF5ixkxu3#m`@8t;j54-yhjvHs; zn4ZAN^dToihgGOAwzuD`=y6WI55~d{zHRd$H$P4V5VnWyRH0O-v@A)!#K@7EDV*c3 z>8nq6T3T=E4fcj>53UW;sv26uHA1_vtsbidcf{P}V{D)E!rsrBG-;BptyjsL!QhRW zaBbN(GxnLaxMqT*F;CZ>%B{DQY-CzyXZdu?yV94|6|oF_OVVN65&4Z;@ks2BvN7Y3 zmuXtn-Z4x!-?7hcHN4bGTGUD8{Y?K?xWC0D+K|-88r-8*uEGAAD}U>5q*G`wsvTHox3#o8Pqyc zhmD<$`7qNK*U~3Xyt=c#iaigL_F0|fntfbou^(-t(lVN|#TM+9q*>*M{oqT{#T5HF^>b_9Z!WwLJs_#fQ z^YBQc&f{tY5Zz`W_Vv}*`~>(-o3uwh@w;aY1u&mn{*dxJdSKWb*N6^gXh|^ znt9F!xKLgL&)+l+Ly`-RQXOxIWM)A&Mf<2W6uFHsDO$K?!wX)kN4&=Qk5_j#5zD&l zhUUz+ryBP5HwBzLaG*Q-j_zey+1Wj?M)0)fT3)ydyHKjv>w8I&0VbVIO=5!$uv6M& z>?=QCN%BXQoR*nKV?oJsEz4T05AZ8cZHw?{pDo$t$ctaSUiXU=ihYw^Wwl0Fq867DbZYi z$Kp^x$)d7S`d`Xw975yt%0xHz5Xk;y{KUIXHA-a=u|t0T&S(0Vscc@5eQCchG-RD1 zC{Bo`D`u>9<2Q>K7f7Tm$_sbSkR$Qi#;Oc_eSHm1ZfMvC`Y1J#%fD$X-a1B0gsIr* z`b!h3>zq}kcL;BMbG7G?8;R(89uO#vC?Ft9H;ua?>^S@t5APlS-Ws1H3}KN-QP|{A z{Io>EbV^N4ZOD*o;OMwDxlomdui8eBKE?T%(2j(MhlBg3X$MUV?756Q7F`d^1FKC; z%VCUJQ3uXa2P|6B z2duk{0DAvGy-x1q#)n^s`0?&;Ve}y6NHhG0)7{h1GArsN}}#>S_!v&`eCx)5B-PSWmB-VghBf6X!&UM51g9 z05Kv@Y$x-jaW0uPc}R73k-$}5&{NzC!!q*U?Le{l_=W(*?+)jlT^uv;90zU(fI}IBK`##t1Z3& literal 0 HcmV?d00001 diff --git a/utils/fsPosts.json b/utils/fsPosts.json index b8e1a89..b18a8e7 100644 --- a/utils/fsPosts.json +++ b/utils/fsPosts.json @@ -1,4 +1,5 @@ { + "skill-doesnt-work-yet": "skill-doesnt-work-yet.md", "runme-eval": "runme-eval.md", "substitute-yaml-with-nouns-verbs": "substitute-yaml-with-nouns-verbs.md", "typed-env-vars": "typed-env-vars.md", From d6513a12f470bb60af309d50211ccb081ebb4c13 Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 14:52:15 -0700 Subject: [PATCH 2/7] Polish generated skill blog introduction Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md index ae6f993..136c6b0 100644 --- a/fsPosts/skill-doesnt-work-yet.md +++ b/fsPosts/skill-doesnt-work-yet.md @@ -19,13 +19,13 @@ canonical_url: "" runme_blog: true --- -You generated a skill. Good. That is the easy part. +Your agent generated a skill. Good. That was the easy part. -LLMs are not generating bad skills because they are careless, or because they misunderstand what a skill is supposed to be. They are doing exactly what they were trained to do: infer the most likely behavior from the context in front of them. Given one repo, one task, and one successful outcome, they will often produce a skill that recreates that outcome beautifully. +The skill can look right. It may even work beautifully for the repo and task that produced it, under the same assumptions. It may have captured the moment, not the method. -That is also the problem. +The model was neither careless nor confused. It did exactly what it was trained to do: infer the most likely behavior from the context in front of it. Give it one repo, one task, and one successful outcome, and it cannot reliably tell which details should generalize and which are specific to that instance. -A generated skill can look correct while being too specific to the moment that produced it. It may work once, in one repo, under one set of assumptions. +It may reproduce the outcome without capturing the broader capability you thought you captured. ## When Agents Generate Context Blind @@ -33,8 +33,6 @@ What makes it sting is that the eval has the same problem: it can certify the ve A good eval should push against the skill: test what it is likely to fake, skip, or overfit. The skill should push back by revealing weak rubrics, missing edge cases, and shortcuts the judge accidentally rewards. That tension is the loop. Each side makes the other sharper. -Let's make that concrete. - ## The Setup Say we generated a dependency-updater skill. From e48402941b8e417f7d6934dc38a9a5431f9ce193 Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 15:00:34 -0700 Subject: [PATCH 3/7] Sharpen skill pressure section heading Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md index 136c6b0..97be419 100644 --- a/fsPosts/skill-doesnt-work-yet.md +++ b/fsPosts/skill-doesnt-work-yet.md @@ -33,7 +33,7 @@ What makes it sting is that the eval has the same problem: it can certify the ve A good eval should push against the skill: test what it is likely to fake, skip, or overfit. The skill should push back by revealing weak rubrics, missing edge cases, and shortcuts the judge accidentally rewards. That tension is the loop. Each side makes the other sharper. -## The Setup +## Put the Skill Under Pressure Say we generated a dependency-updater skill. From 6a96c26dd067235543ec5f8fae915f173186fd64 Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 15:10:58 -0700 Subject: [PATCH 4/7] Clarify generated eval feedback loop Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md index 97be419..7d25a21 100644 --- a/fsPosts/skill-doesnt-work-yet.md +++ b/fsPosts/skill-doesnt-work-yet.md @@ -29,9 +29,9 @@ It may reproduce the outcome without capturing the broader capability you though ## When Agents Generate Context Blind -What makes it sting is that the eval has the same problem: it can certify the very overfit it was supposed to catch. If the same model generated both the skill and the test, the eval is not yet an impartial judge. That is like asking a team to referee its own game. +What makes it sting is that generated evals have the same problem: they can certify the very overfit they were supposed to catch. If the same model generated both the skill and the test, the eval is not yet an impartial judge. That is like asking a sports team to referee its own game. -A good eval should push against the skill: test what it is likely to fake, skip, or overfit. The skill should push back by revealing weak rubrics, missing edge cases, and shortcuts the judge accidentally rewards. That tension is the loop. Each side makes the other sharper. +Think of the skill as the unit under test and the eval as its test suite. The eval should expose where the skill fakes, skips, or overfits. In turn, surprising passes should reveal weak assertions, missing edge cases, and shortcuts the eval accidentally rewards. That tension is the loop. Each side makes the other sharper. ## Put the Skill Under Pressure From 33e1c8827ce9ef5677c9c8ab1e64cfcb01fd5fce Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 15:30:28 -0700 Subject: [PATCH 5/7] Sharpen blog excerpt hook Co-Authored-By: Claude Opus 4.8 Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md index 7d25a21..859d0b3 100644 --- a/fsPosts/skill-doesnt-work-yet.md +++ b/fsPosts/skill-doesnt-work-yet.md @@ -5,7 +5,7 @@ post_author_avatar: "sebastian-(tiedtke)-huckleberry.jpg" post_author_url: "https://twitter.com/sourishkrout" date: "2026-07-21" post_image: "skill-doesnt-work-yet-cover.png" -post_excerpt: "A generated skill can look right while overfitting the moment that produced it. The fix is a tighter eval loop where the skill and the judge make each other sharper." +post_excerpt: "A generated skill captures a moment; an eval loop turns it into a method. The catch: generation feels like the finish line, so the loop never gets built. Here's how to pressure-test it." post_slug: "skill-doesnt-work-yet" tags: ["developer tools", "ai", "automation", "skills"] category: "runme" From e3d2d46bfa3e12d99793befa265653d778eef5a4 Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 15:33:17 -0700 Subject: [PATCH 6/7] Refine skill post wording Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md index 859d0b3..a072132 100644 --- a/fsPosts/skill-doesnt-work-yet.md +++ b/fsPosts/skill-doesnt-work-yet.md @@ -21,7 +21,7 @@ runme_blog: true Your agent generated a skill. Good. That was the easy part. -The skill can look right. It may even work beautifully for the repo and task that produced it, under the same assumptions. It may have captured the moment, not the method. +The skill can look right. It may even work beautifully for the repo and task that produced it, under the same conditions. It may have captured the moment, not the method. The model was neither careless nor confused. It did exactly what it was trained to do: infer the most likely behavior from the context in front of it. Give it one repo, one task, and one successful outcome, and it cannot reliably tell which details should generalize and which are specific to that instance. From 1fc27be3cd009ad6d34a790d32d0a1582e77cf8d Mon Sep 17 00:00:00 2001 From: "Sebastian (Tiedtke) Huckleberry" Date: Tue, 21 Jul 2026 15:39:26 -0700 Subject: [PATCH 7/7] Polish skill post excerpt Signed-off-by: Sebastian (Tiedtke) Huckleberry --- fsPosts/skill-doesnt-work-yet.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fsPosts/skill-doesnt-work-yet.md b/fsPosts/skill-doesnt-work-yet.md index a072132..186d805 100644 --- a/fsPosts/skill-doesnt-work-yet.md +++ b/fsPosts/skill-doesnt-work-yet.md @@ -5,7 +5,7 @@ post_author_avatar: "sebastian-(tiedtke)-huckleberry.jpg" post_author_url: "https://twitter.com/sourishkrout" date: "2026-07-21" post_image: "skill-doesnt-work-yet-cover.png" -post_excerpt: "A generated skill captures a moment; an eval loop turns it into a method. The catch: generation feels like the finish line, so the loop never gets built. Here's how to pressure-test it." +post_excerpt: "A generated skill captures a moment. An eval loop turns it into a method. The catch: generation feels like the finish line, so the loop never gets built. Here's how to pressure-test it." post_slug: "skill-doesnt-work-yet" tags: ["developer tools", "ai", "automation", "skills"] category: "runme"