here is a short tldr of the topics i will move through in some way shape or form.

• the death of the cyber hype cycle
• VR was accepted only while the hype cycle was alive and everyones true colours are now coming out
• you cannot prevent a 4th/5th/nth upstream party selling their credentials to an IAB
• the entire world has been poisioned by hyper gamification and this is why you all love vibe coding so much
• bug bounties are dead, (maybe) long live bug bounties

its been about half a decade now that my title has the word security in it, but its been about 15 years and counting that my job has involved security. i have said this for years to anybody that will listen and even beating those that wont over the head with it; cyber and security are nothing but functions of IT. they are not seperate domains, you are not the military with your 5 planes of battle space. app control, network segregration, removing the any/any rule in the fortigate that makes your myob work (which it DOESNT! the activation server was just down and came back up when the rule was added, it was a coincidence!) all belong to IT.

there used to be a literal wizard with a beard about 3 foot long who could tell you exactly how to set up the solaris server correctly so a bridge is made to interface between the superannuation program running on the machine literally under his desk and the timesheet results from CICS on the mainframe via RACF. this man, knew all about security. this man, was always part of IT and did not have a seperate segmented zone to sit in with some cybery title. he was IT security.

and somewhere along the way, perhaps we can blame crowdstrike, i dont know and i dont really care, marketing and perception forced some sort of schism; IT became janitorial and cyber became the new hotness, the sexy thing to do when you want to make 200k a year despite not knowing how to spell the word computer and perhaps most tellingly not actually giving a flying fuck about computing.

and it becomes cool! it becomes trendy! wow ethical hacking! beating the bad guys by being a bad guy (despite almost NOONE a) paying for a real red team or b) being comfortable with what actually goes down in a real red team and running it back to a PT where you dont even touch the sides of real bad guys)! no not that soc stuff, thats janitorial! thats IT! get me out of there and into the red team! and so you end up with people moving from sales into pentesting and being shocked and surprised that theres an ENORMOUS trove of prereqs to absorb to not be dangerous.

cyber for all! everyone wants cyber! yes cyber for me please! yes please give me a red team engagment! wait what the FUCK do you mean you are going to deploy deepfake voice cloning of the ceo to trick helpdesk into resetting his password? how dare you even THINK of forging tokens of the soc analysts and deleting your own logs out of the siem? in what world do you think i want you to shim the locks to the facility and drop an implant? Yes i know i asked for a red team but this is just unacceptable! this is criminal!

this is criminal. yes, the final retort of coomputer related endvevours that result in another party losing face. when clients balk at the reality of what activities previous succesful red team engagmeents undertook, its always a two sided coin where some varient of “this is illegal” (thats against our policy, that is unorthodox) is backstopped by the real issue of “if you do this and are succesful, we will look like idiots”. i dont think the coin analogy fit but my head hurts and im not thinking of a new one. we will be coming back to this later.

there is an absolutley ENORMOUS chasm between customers and consumers of offensive services who embrace shift left and integrate consultants as part of their lifecycle, and those who are amazed and upset that you cannot simply walk up to the front desk and order “1x pentest please” for 3 days across a multi forrest domain and have it meaningfully represent ANYTHING, as well as being amazed that we will not infact give them legal documentation saying you are unhackable to pass to their insurance teams afterwards.

and that second group of clients reveals an unpleasant truth about the hype cycle; offensive security during the hype cycle was only ever allowed to exist via polite lip service due to the changing market forces making it sexy and hot, and overwriting the REAL thoughts that everyone actually finds it offensive (hue) and criminal.

approx a year ago i ranted on this particular topic in short form, but more aimed at individuals calling for the criminalisation of offensive tooling // vuln research, but now, today, microsoft has shown their real stance. they know the hype is dead, and they know, because they always understood it, security IS janitorial in nature. mop the floor, clean the spill. lock the doors, change the keys. pentesting is nothing but QA - hows THAT for a sexy life changing career you should get into to make millions and not have to deal with stinky computers?

and microsofts latest post calling for the deployment of the cyber police in a threat somewhat reminscant of the jessi slaughter saga but infinitely more serious and likely to have real world ramifications, brings us full circle to the 2000s. this is criminal, cry ms at being made to look the fool for a string of vulns they ignored. “how dare a measly researcher find these things that we, microsoft, titan of industries, missed? how foolish do we, micrsoft, look in the face of ever escalating pressure to adaquetly respond to security research but instead choosing to deny people their livelihood by marking things as not violating a security boundary, listing research as NA and then issuing a patch, threatening researchers for disclosing the CVE number for a finding? why wont these criminals just stop!”

“criminals, criminal criminals! criminals committing cyber crimes causing computer concern! chicanery! dont these researchers understand the world would be safer if they all just stopped this activity? if everyone in VR just stopped looking for vulns, there would be no more exploitation!” - the dastardly cry of the utterly derranged.

but as i said before, this mentality has been leaking out of the cyber circle jerk for a while now, with people looking to try and score cheap brownie points by having the same opinion on internet saftey as their favourite conglomorate in a desperate, exasperated attempt to retain employment in the positively fucked market environment. its just now orgs have no need to pretend anymore; VDP budgets are being dried right up, literal threats being deployed, teams being excised but above all an enormous desire, compulsion, NEED for these billion and trillion dollar orgs to save face by blaming the existence of any issues in their products on the discoverer of said issue.

exactly how far removed from a functioning brain does one need to be to be physically unable to reach the conclusion that if a sole operator with a laptop can find these vulns and weaponise them, obviously real threat actors and actual criminals are finding them too? “but it makes it easier for criminals!” except it toally does NOT. am i going to a) keep abreast of what is new and popping in sploit land when im not being paid for it as part of my job in IT, or b) hit up my local access broker for some valid creds to an MSP?

and so we move again into an observation i have made year after year anytime i have to talk to C levels about threats and the landscape; people selling their creds is an unstopable problem. it cant be solved. theres no measure that can be installed, policy written or procedure theatrically played out. all one can do is mitigate the damage it will cause (do note i said WILL and not CAN, because somewhere along the line, one of your (yes YOU personally) upstream dependencies will have their own upstream depenency that will have someone happily trade their helpdesk creds for $5000. cause if youre making 3/5ths of fuck all at the age of 19 at a job you hate almost as much as you hate yourself, as if you wouldnt make that deal.

all you can do is make sure your garden is as well tended as it can be; you have segmented properly, you have POLPd and PLOPd and POLOd and YOLOd, you have minimised the blast radius. one may be wondering how this ties back to shouting at m$, and the answer is, using metrics i have procured from deep inside my own anus, significantly more damage is being done with any of your random supply chain attacks that crop up basically every single day of the past few years than any (and realistically, all) of Nightmare Eclipses releases. because someday, all of these will be patched and mitigated. there is nothing you can do against the eternal threat of a credential sale by a human being you in no way shape or form have control over.

there is also nothing you can do when a developer of the paas platform your paas platforms developer uses gets fucked by a malicious roblox mod the first platforms dev put onto his work laptop, with excessive share perms via ANOTHER platform leading back to the paas platform you actually use. make sure to read that carefully and see how many steps away from you, the innocent end user, this is, and how you, the innocent end user, can do ABSOLUTLEY NOTHING about this.

but, its all the fault of offsec. if only a tool doing what psexec has done for decades didnt exist on github, this developer would have never been able to download a backdoored game mod onto his work laptop!

vdp payouts being lowered, companies acting openly adversarial to researchers and testers now the hype cycles fall out of favour, programs being closed in favour of putting that money into tokens, researchers publically discussing brokers instead of the traditional bounty hunting process and it appears the industry on both sides might be sick of bug bounties. one of the largest disparities i have come across in pentesting vs hunting, and this should be stunningly obvious to anyone whose made it this far, is pt clients care far, far more for a remediation plan than being shown impact.

and i think this might be where we are at; the traditional model has reached an inflection point of “so what”. “oh, you found another lpe? just chuck it on the pile of other lpes weve got to work through.” what a world to live in where an entire new attack class has opened up via the frag lpes and its been a somewhat collective shrug after the 4th variant dropped.

i think the change may be somethign more akin to what geohotz was trying with tinygrad where you get paid for PRs you submit that fix shit. “woah you want people to do m$ work FOR them?” what do you think all this free vr is? they arent getting paid for it anyway with the trash handling of issues, so perhaps the answer is to pivot to doing the work for them that they WILL pay for.

it is time to round this post out and lie the fuck down because i am very, very ill and acciedentally coughed on the amazon delivery driver bringing me my copper tape that i am using for another upcoming project; diy 3d printed pcbs. astute readers will note i really not cover the ai topic, and truthfully that sentence in the tldr encapsulates it perfectly and b, if you simply read my old gamification article and mentally regex in the word ai instead of fisher price platform, the entire article stands perfectly well.

Following the evolution of the twitter posting yugioh bot and we can see it went from using groq keys before they banned me (as of 8/10 2025 I am still banned), to a llama model on my nuc, to a gemma model on my nuc, to a gemini model on the cloud, to a qwen model on my raspberry pi. From the cloud back to the ground, back to the cloud and back to the ground again for what I assume will be the final time I configure that tool.

This sets the scene (somewhat) for how a few of my tools have lived - running off a cpu only instance of llama and or gemma (depending on what month it was) on my nuc. It is december 2024, and I am cranking out post after post and internal tool after internal tool in a manic panic to accomplish things I set out to do earlier in the year and didnt get around to because I was trying very hard to not kill myself. It is december 2024.

After being very, very angry at being banned by groq I decided everything needed to be local, and set out to formulate my own tooling to solve issues and replicate products that are censored. I havent tried it again since then so this may be incorrect information now, but at the time the gpt web browser would for instance refuse to have a look at 4chan. I decided it would be a good idea to build out my own scaffolding to scrape web pages in order to have summaries sent to me. This was partially born from two realisations; I spend far too much time online, and, much like my api key debacle, I will probably get banned for wrongthink at some point in time for simply using the tools in a manner deemed unacceptable (ie putting them over links and articles unfit for corporate consumption).

First plain old beauitfulsoup was employed, and it worked ok, some of the time, sometimes. Sites that use heavy amounts of javascript, like 4chan, refused to load properly and as such were not able to be scraped. Moving to selenium, I was then able to parse the posts in a random /x/ thread and receive summaries of it after chucking the raw parse to an llm. Progress!

But then came sites that put content in particular tags. I will not be elaborating upon this because it was almost a full year ago and I dont recall exactly what the issues were and the artifacts I retained were only the final versions, not the progressions I am describing. This necessatated the combination of BOTH selenium and beauitfulsoup, as I couldnt work out how to control the selenium api enough to get the sort of control I am used to working with with beauitfulsoup (thanks to time spent refactoring the black hat python repo from p2 to p3 back in 2020 (you will have to believe that I did it, the repo has been deleted since the 2nd edition of the books come out, making it unnecessary to keep around (I do need to further my no starch press book additcion and get the 2nd ed))) so I had a strange mechanism that would first try a selenium pass, and if that failed to produce any content move to a bs scrape.

I didnt like what bs (hue) this had turned into, and in the bath researched what alternatives to selenium existed. Every now and then I am surprised who releases foss tools of note, and in this instance I am refering to microsoft and playwright. an excellent tool, but necessetated me to throw out all I had put together thus far and begin anew, which ended up being the right move because it fucking works. I say works, in the current, present tense, because the code I and various gpts (including local ones!) cobbled together between december 24 and the 2nd of jan 25 when I hoisted it onto its permament home, is still intact. The only changes its received are context length expansions when I found gemma 3 4b was just as smart as llama 3.1 8b and lets me have 2x as much context for the same ram usage, and a reduction in the amount of articles I get per day. Everything else, has been running smoothly and perfectly except when telegram has an outtage.

Wait its sending to telegram? Yes, because basic authentication to emails is dead and I was not going to fuck around to get 2fa and or oauth via imap working over a python script. The past 10/11 months have shown that telegram is a perfectly acceptable distribution channel for this anyway!

I then find out, a week after its finished, ironically through a summary email, probably the grugqs, that someone else had infact released this entire project online and would have saved me the effort. This displeases me somewhat.

Here is a small tally of the artefacts produced. This list will be updated as the story continues towards its laboured and confused conclusion.

•web scraper
•ability to “see” contents of webpages

It is january 2025. Rumblings begin about this “deep research” business. It sounds pretty fantastic, and the next logical step of the above project; instead of providing the product to distill, have it go and RETREIVE its own product and THEN distill. Everything that had been mentioned about it at this point indicated it was (and it indeed turned out to be true) going to be gatekept behind the $200 USD per month subscription. So a line entry was put onto The Whiteboard (TM) saying i was going to build out a deep research tool of my own. Not having used one i was not too sure exactly what it needed, but figured being able to “see” webpages was a great start, and perhaps some sort of OCR capability to read PDFs?

A second function is added to the website; to ingest pdfs, run an ocr tool over them, take each page and summarise, and then reconstitue these summaries into a cohesive whole. I get this working over a few days, grappling with issues with my dev machine and my prod server having different python versions, yay! Eventually this is sorted, and the functionality is live. It works, and it works fucking WELL. Initially this is NOT pointing to the nuc cpu llama, as I need it to be free sometimes to process the yugioh shitposts, and instead is pointed at my 3060 dual weilding machine. You may have guessed, based on the previous conclusion and the preamble, upon completing this, I was exposed to another freshly made ready to go solution. The bad vibes continue to pile up.

the tally so far sits at

•web scraper
•ability to “see” contents of webpages
•ability to “see” contents of pdfs

It is february, and I have my nice lime green yeti from my wife and my new ebook reader. I quite like my ebook reader. Being eink however, it does have some problems with webpages. Not every web page can be properly scraped by my toolset, because I am limited by the context length of the llama 8b model I am hosting; some pages are simply too large. I have a clunky sort of check mechanism in place to validate its under 30k tokens, this works and prevents it fucking out and requiring me to ssh into boxes id rather not touch. So if a site is too big to be summarised, this means its too big for me to read on the computer (I dont like READING on the computer, I SKIM fine, but I want somethign nicer to my eyes for long haul reads (which is ironic because editing this inside darkmode github is doing bad things to my sight)). So a new addition to the site was generated that will take a website, make a pdf of it and then crunch that through to an epub. It works well but I was really kidding myself cause most of the epubs I have generated I still havent read. cest la vie.

thus far we have

•web scraper
•ability to “see” contents of webpages
•ability to “see” contents of pdfs
•ability to take sites too big to parse summaries from and crunch to pdf -> epub

It is march and I am talking a lot about using ai to aid in testing. And I do, in indirect ways. I have no interest in feeding entire dumps of traffic to the tool; as an experiment I tried this and it fucking CHOKED, was a very pointless afternoon. it is march, and i decided to take burpference to the nth degree and cobbled my own config together to get it to point to my lmstudio api. Escaping all the shit that appears in raw traffic logs was a nightmare. It languished on the ever present to do list for a while, because I could feel that with some more work, it was actually going to be an insanely useful tool. Surgical, targeted precision instead of asking the poor little palantir to eat the firehose

Imagine my disappointment (yes I AM that conceited, how did you know?) when a few weeks later burp ai drops. And it does what i was lookign to do, but within the ui itself instead of via a second rate plugin modified by a third rate hack (me). Once again I am beaten to the punch, but this time I choose to not see my experiments through to their conclusion and throw them out.

we have obtained at this point

•a web scraper
•a tool with the ability to “see” contents of webpages
•a tool with the ability to “see” contents of pdfs.
•ability to take sites too big to parse summaries from and crunch to pdf -> epub
•a real hard lesson in the differences in compute between my 3060 and real inference g/tpus

It is may, and I get around to testing the gemma models. These have vision capabilities, but lmstudio does not. I want to be able to feed the model an image and have it describe it and or read whats on it if needed. This feels like another step towards the diy at home deep research capes I am still planning on building out. Its felt like a “missing piece” for quite a while now; gpt has had the ability to see images for ages, why not the at home models? So a new api is written up, that will ingest an uploaded image and retreive a description from gemma. It would be nice for it to keep it in context however, not just discard it. This is a piece i am struggling with, and put the project down for a while.

we have obtained at this point

•a web scraper
•a tool with the ability to “see” contents of webpages
•a tool with the ability to “see” contents of pdfs
•ability to take sites too big to parse summaries from and crunch to pdf -> epub
•a real hard lesson in the differences in compute between my 3060 and real inference g/tpus
•a half baked api to truely look at images

It is now june. we have a list of compostable pieces; with enough string and glue, surely we can take all of these seperate pieces and produce a functional deep research agent. It is june, and chatgpts deep research has been released to the plus subscribers, one of which is me. This is not demoralising; it is infact more impetus to get this project working, as I am blown the absolute fuck away by how good the product is and burn through my queries often. Nothing, and i mean fucking NOTHING i have used up to this point (STORM is the only name I can recall from my testing periods) has ever been able to properly address my test question about the yugioh ocg format in the late 90s. its esoteric as all fuck and its intentionally designed to trip the bots up. This is due to the inclusion of three letters - OCG. I will perhaps elaborate in another post another day.

Surely, I have enough pieces to string together some sort of research model, right? It can get webpages, read them, get summaries, do the same with pdfs, format sites too big to parse normally, and look at images? This has got to be all thats required to get somethign semi useful up and running (and look future me genuinely agrees - cobbling them together WOULD produce some results of note for sure). But I am exposed to the project local deep research and wouldnt you believe it it actually fucking works. So once again, I am thoroughly beaten to the punch, and become reasonably disappointed again that my work would be pointless upon this task. It then turns out I would spend about as much time as I figured it would take to BUILD something out trying to TROUBLESHOOT this solution - perhaps this is an indication I underestimated the effort required to build this project myself by several orders of magnitude..

•a web scraper
•a tool with the ability to “see” contents of webpages
•a tool with the ability to “see” contents of pdfs
•ability to take sites too big to parse summaries from and crunch to pdf -> epub
•a real hard lesson in the differences in compute between my 3060 and real inference g/tpus
•a half baked api to truely look at images
•a functional deep research tool for when i have burnt my OAI queries

It is september, and I am looking once again at RAG solutions. LM studio has an update, and it adds rag functionality…as well as this little icon. lm image upload

Two guesses what this does; why yes it allows injection of images into the current chat! Unlike my api, it will not call a new chat with each image uploaded. It is september, and I for once, feel relief. “Thank fuck” I say, “thank fuck I no longer need to build out that functionality”. Its here and its been made for me by the hardworking devs who stay on the bleeding edge to provide to me, those “"”only””” on the cutting edge. “You were beaten to the punch FIVE SEPERATE TIMES, how are you on the cutting edge? have you no shame?” No, I dont have any and yes the fact I am even in this sandpit means I am on the cutting edge. Do you know how hard it was not to be condesending when I heard the principal of another team talk about fine tuning a domain related model for his teams use? Thinking he was going to use our internal hash cracking rig for it? Do you not understand I went down this route 18 months ago and worked out we lack the hardware for this?

I have referenced multiple times the concept of agents being bash loops. This is not a joke to me, this is literally how my site and helpers run, off a bash loop and a cron job. Expanding any further here will be trodding already trodden ground, read about me lording over people with my big brain and my big penis here

Back to the main point here; unless you are actively invested in pushing the boundaries and being a frontier in your respective domain, there is little ADVANTAGE to being on the bleeding edge. The truth of all my little experiments, despite their seemingly disapointing conclusions (because for some reason I attributed being a super special unique flower with providing me satisfaction) the undeniable reality of it is I built them because it was fun. The fact they served a purpose is incidental and tangental; these pieces are only getting mentioned because they are the ones actively in use today - other pieces suchg as the novel drafting toolkit and the barebones of a pokemon playing agent and a series of jwt tools in a workflow were built out; all of these sit in a folder called ‘zzz_requires sorting’ where they will probably be deleted once it has been over 12 months since I last touched them.

Its fun to build things. Its exhausting racing to be first all the time. It is also impossible to be across every piece of new research from every lab. Id rather spend time in my own lab tinkering with stupid shit and being delighted and building for the fuck of it. Because ultimately, in what has to be the most cruel joke to those chasing the bleeding edge, time will erode that edge. If I ignore what is happening in The Field (an interchangble placeholder for ml/infosec/whatever it is that has my attention this month) for any period of time, whatever is at the forefront when i return will have overrode whatever was the SOTA that I missed while I was away, almost as if it never happened.

This has been a somewhat cleansing musing and in some ways feels spiritually like the end of year summaries I write into my sigil book every year. Things didnt really go to plan most of the time, but only because I have the freedom to potter and fuck around with little doodads and build oddities was I able to even miss the mark. There is a quote I saw I very much like; “you waste years not being able to waste hours”. I will not elaborate any further, but end this with a koan - if I wasnt actively building out the toolsets I was supplanted by, would I have ever known about their existence?

I forged a key and found the door already open. Had I not forged, would the hinge have shown itself?

And this is very apt and true about compute; the more of it you have, the more of it you end up using, therefore requiring more compute, which will be used for things and so on and so forth this goes on until you realise you have almost a terabyte of RAM in your house. There is another project coming that will capitalise on this idea even further, as I am actually running out of power points and am begining to worry a little that my garage might not be safe if I draw any more watts into it.

I know I still havent gotten around to putting the post about my AI agents aka python and bash loops up yet, and of course this post indirectly references them, I am not avoiding it I just havent gotten to it, because truthfully they dont feel finished yet despite the core functionality of them being static since..well about this time last year.

But some of these loops are processing a gemini instance on my nuc. And they require, for some tasks, a metric fucktonne of ram (over 40% of the NUCs available ram!) so I can load up the appropriate context length. But one of the tasks that this ever loaded LLM is performing is writing very bad twitter posts about Yu-Gi-Oh! cards. Since both the llama 3.1 8b and the gemma 3 4b were not very well versed in esoteric deep cuts (look its not their fault, I did set the card pool the script cycles over to only be GOAT format and prior; this is knowledge that is two decades old) I had a line in the prompt saying if you dont know the card, make something up. And since it takes up a lot of my compute to just make up bullshit, I wondered how far I could push it. Whats the smallest device and model I can use to produce semi coherent posts?

Turns out the answer is, as you may have guessed from the title of this post, my old rapsberry pi 3b. Man I have had that little guy for a long time now. In 2017 it used to host a neat little project of mine that would check my WAN IP every morning before I woke up and if it has changed, it would update my OVPN file and email it to me so I could get back on my VPN. Anyway. The LLM of choice (chosen by our robot overlord in a stroke of pure irony) is Qwen 2.5 0.5b. 500m params. Seems small doesnt it? Yet thats still enough that I would need an A100 running for a few days to train it, which is most certainly not going to happen.

So how do we do it? Pretty easily tbh. This is a ready to go recipe that requires no fucking around or funny stuff. Compile time is just over 24 hours.

Install dependencies

sudo apt update && sudo apt install -y git build-essential cmake python3-pip libcurl4-openssl-dev

Increase swap so the linker & runtime don’t choke

sudo sed -i 's/^CONF_SWAPSIZE=.*/CONF_SWAPSIZE=2048/' /etc/dphys-swapfile
sudo systemctl restart dphys-swapfile

Grab repo and compile

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DLLAMA_NATIVE=ON
cmake --build build --config Release -j2

Install more deps and put on path

python3 -m pip install -U huggingface_hub --break-system-packages
export PATH="$HOME/.local/bin:$PATH"
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc

Download model. It will put a model folder where you are currently.

huggingface-cli download Qwen/Qwen2.5-0.5B-Instruct-GGUF qwen2.5-0.5b-instruct-q3_k_m.gguf \
  --local-dir models --local-dir-use-symlinks False

Test. You will note that I did not follow the advice above and moved to the models folder when downloading and now need to call it from the models folder inside the models folder. root@raspberrypi:/home/pi/llama.cpp# ./build/bin/llama-cli -m models/models/qwen2.5-0.5b-instruct-q3_k_m.gguf -t 2 -c 128 -b 32 -ub 16 -n 80 -ngl 0 --temp 1.2 -p "Please give me a piece of trivia about the Yu-Gi-Oh! card 'Barrel Dragon' in a terse and factual style, with hashtags. Ensure it fits within 240 characters. If you do not recognise a card or do not have any trivia about it, fabricate a piece of convincing Yu-Gi-Oh! trivia."

Lets host an API so we can query it

cd /home/pi/llama.cpp
./build/bin/llama-server \
  -m models/qwen2.5-0.5b-instruct-q3_k_m.gguf \
  -t 2 -c 128 -ngl 0 --port 8080 --host 0.0.0.0

We best set it up so it runs on boot. Create a service at /etc/systemd/system/llama-server.service and populate it as below

[Unit]
Description=llama.cpp server
After=network-online.target

[Service]
User=pi
WorkingDirectory=/home/pi/llama.cpp
ExecStart=/home/pi/llama.cpp/build/bin/llama-server -m /home/pi/llama.cpp/models/qwen2.5-0.5b-instruct-q3_k_m.gguf -t 2 -c 128 -ngl 0 --port 8080 --host 0.0.0.0
Restart=on-failure

[Install]
WantedBy=multi-user.target

Install it and start it.

sudo systemctl daemon-reload
sudo systemctl enable --now llama-server
sudo systemctl status llama-server

Hit it remotely

curl -s http://10.1.1.156:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model":"qwen2.5-0.5b",
    "messages":[
      {"role":"system","content":"You are a Yu-Gi-Oh! card trivia expert."},
      {"role":"user","content":"Please give me a single sentence of trivia about the Yu-Gi-Oh! card zz45 in a terse and factual style."}
    ],
    "max_tokens":50,
    "temperature":0.7
  }' \
| jq -r '.choices[0].message.content'

You will note I asked it for data about a card that is nonsense to see how it goes making up factoids about things it has 0 knowledge about. I have set the max tokens very low because this one isnt smart enough to understand I need it to fit into a tweet, so I just cut it off rather than massage the prompt like I was doing with gemma.

Here is one that I quite like. Good job little rpi!

It feels pretty good to breathe new life into my doodads. I have even printed it a neato little home, replicating the power mac g5. I get somewhat nostalgic for that body shape; I used one of these in my music production units when at uni. Those were some of my absolute favourite units to do and I learnt an absolute shitload. Here is its happy home inside my happy home.

In order to prove the adage we opened the post with, I will vaguely allude to one of the projects I am intending on putting onto the nuc with the compute that has become available now I am not hoisting an LLM every two hours into memory since the external gpu has other tasks (another thing I need to blog about, jesus they just pile up). I have become vaguely aware of this Diablo 2 project . I was very very very close to buying a 96GB nuc to supplement my current one, but I KNOW I will fill it too and “"”require””” a third one, when what I really need is a set of APIs to turn things on and off again…

“In the context of machine learning, a “transformer” refers to a type of neural network architecture that excels at processing sequential data, such as text or audio, by leveraging an “attention” mechanism”

Well that certainly technically qualifies as an answer, but raises more questions! Namely, what is attention?

“Attention is a machine learning method that determines the importance of each component in a sequence relative to the other components in that sequence.”

OK, this makes some sense, sounds like how speed reading works (here is a good blog on it I paroosed some time ago from Tim Ferris that i can confirm does work (but i dont do it, i find joy in reading and do so slowly)). Essentially in any given sentence, there are words that provide more value to the overall meaning trying to be conveyed. Attention, in the ML sense, is a method used to do the same thing, work out which words are important.

This is pretty interesting; when we are taught to read there isnt a list of words that mean more than others at all times, its contextual. For instance: “a cat shat on my hat” vs “my cat shat on the hat”. If we look at the words my and a. Their placement in the sentence structure changes it entirely; one is a personal tragedy and the other is an amusing anecdote i would chuckle about, respectively.

I do not nor do I want an accessiable picture of a cat shitting on a hat, but instead enjoy my cat wearing a hat.

We are not taught that “my” or “a” holds more weight as a word, ie that instances of my are more “powerful” or deserve more “attention” (you getting it now?) than the word a, which is in itself another branch on this metaphor; a is a (lol) word and a letter and in some cases, a number. It is all contextual, and the models learn this context dependancy in the same way we do; by reading the current input and applying meaning at real time, unlike say linear regression where the model learns that the coffecient is equal to w and will always be equal to w, no matter the inputs.

But how does the NN do it?

Sentence is inputted.

Lets use “my cat shat on a hat”

The sentence is broken down into tokens.

How tokens are defined depends on how the creater of the network defined it. For this instance, we will use words as tokens.

This would look like ‘my’ ‘cat’ ‘shat’ ‘on’ ‘a’ ‘hat’

Each token is mapped to a unique numerical identifier based on the models vocab.

In this instance, we will pretend the mapping washed out like so (for illustrative purposes it is irrelevent).

'my'	27287
'cat' 	28498
'shat' 	47385
'on' 	7265
'a' 	27020
'hat'	3828

Each token id is converted to a dense vector (aka an embedding) to capture the meaning of it within the sentence.

Wait, what is an embedding? A trainable lookup table where each ID in the vocab corrpospondes with a row, and each row is a vector (array/list) of floating point numbers.

Vectors are randomly initialised at the start of training. During training, the model updates the vectors with similar usage patterns end up with similar embeddings. ie cat/dog might become closer, hat and helmet, shit and shat, onwards and upwards. The number of floats in each row is equal to the embedding size (which you set prior to training). For example the old BERT uses 768. Our fake example will use 3. Dense means that each element in the array must contain a value, unlike say IPV6 where you can just set :: and omitt part of your address. Elements in the array are usually (ie 99.9999% of the time) between -1 and 1.

'my'	27287		0.23 0.44 -0.008
'cat' 	28498		0.72 -0.45 0.33
'shat' 	47385		0.48 -0.92 0.16
'on' 	7265		0.71 -0.33 0.04
'a' 	27020		-0.77 0.88 0.29
'hat'	3828		-0.65 0.01 0.63

Once we have the words vectors from the table, we assign a positional encoding vector. This is important because despite the sentence being my cat shat on a hat, the vectors are not passed sequentially to the transformer; they are processed all at once! Without tagging the vectors to ensure they are reassembled in the right order (think TCP) we could end up looking at the wrong sentence, ie on a hat shat my cat (ok bad example because that actually still works lmfao)

These are not tacked onto the end, a bunch of functions are performed against the inputs. I am not going to detail them because I a) dont understand them and b) as people smarter than me have said unless you are actually looking to reinvent the field, there is little benefit to getting into the weeds of the maths. I guarantee you Lewis Hamilton has 0 idea how the craft he pilots works, yet he is still able to hold the record for most F1 wins, ever. You dont know all the details about how something works to push the limits of what is possible with it.

Our new table becomes (once the not described math is applied)

'my'	27287		0.23 0.44 -0.008		0.23 1.44 -0.008  
'cat' 	28498		0.72 -0.45 0.33			1.56 0.09 1.17  
'shat' 	47385		0.48 -0.92 0.16			1.39 -1.34 1.07  
'on' 	7265		0.71 -0.33 0.04			0.85 -1.32 0.18  
'a' 	27020		-0.77 0.88 0.29			-1.53 0.23 -0.47  
'hat'	3828		-0.65 0.01 0.63			-1.61 0.29 -0.33

Attention Calculation is calculated.

The transformer doesn’t just stare at the word vectors and guess what’s important. It projects each word into three new vectors using mini neural networks (well, linear layers):

Query (Q): What am I looking for?
Key (K): What do I offer?
Value (V): What info do I carry?

For every word, the model compares the query of that word with the keys of all other words using dot products. A dot product is a math operation between two lists of numbers that gives you a single output telling you how similar the two lists/vectors are. This tells it how much attention to pay to each other word.

Then it squashes those comparisons with a softmax to get probabilities. A softmax is where we turn a list of numbers into a probability distribution, which is a list of values between 0 and 1 that all add up to 1. I vaguely recall this shit from a stupid uni class I did in summer semester because I was behind in credits. Basically things that are more probable get weighted higher (almost stunningly obvious here). It uses theese probabilities to blend together the values, creating a new vector that mixes in info from all the other tokens depending on relevance.

To recap, in this step we are comparing each word in the sentence against all other words in the sentence to tell us how similar they are, so we know how much attention to pay to particular pairings. Then these comparisons are squeezed into a pd so the sum of all the compariosns equals to 1. Likely pairs bubble up to the surface, as unlikely pairs are weighted poorly.

Layer normalisation is applied.

Vectors are rescaled for each word so its dimenaions have a mean of 0 and a variance of 1. This keeps the inputs to each layer well behaved and stable, preventing exploding or vanishing gradients (use your noodle you can work out what is meant) as the data flows through deepr layers. I will not be explaining how this is done as I quite frankly do not care.

Multiple heads (processes running in parallel w/ diff settings) process the input sentence to in a sense ‘decode’ the meaning.

One head might look at how shat and hat are related, one might look at my and cat. Each head learns a different relationship. The outputs from the heads are concatted together for a final output.

There is a relationship between the size of our vector table and how many heads will be used; the embedding size (aka hidden size) % (modolus) head_count == 0. Basically the number of heads modulused against the hidden size needs to equal 0, there cannot be any remainder. If you have an embedding size that cannot modulus to 0 with any number, you need to change the number of heads, or the hidden size (embeddings) or both.

Since our example has an embedding size of 3, we can use 3 heads, or 1 heads, since 3%3 and 3%1 are both clean, but we can NOT have 2 heads. In reality this is not a good number to use; each heads going to have just a single dimension to look at. A good number is 64 dimensions/embeddings per head. If we look at BERT again, 768/64 == 12. 768 % 12 == 0. Like magic.

The output from the heads is passed through a feedforward neural network.

This is a tiny nn with a ReLU in the middle. Which is Rectified Linear Unit. It basically, not to triviliase it too much, takes any negative inputs and returns 0. Its kinda like an inverse compressor; where a compressor will take anything too positive and crush it to x, ReLU will take anything too negative (ie negative at all) and push it up to 0.

Why on earth would we do this? Without a non-linear activation function (like a ReLU) our little FFNN will, by virtue of NOT being non-linear (ie BEING linear) is just going to stack its linear operations ontop of each other, bigger and bigger, upwards and upwards, like a straight line. This is nice for things such as measuring the relationship between sulphur presence and demonic entities but things like words, semantics and meaning, context, are not linear, simple things to understand. So we need to make our functions “non simple” and able to repersent non linear patterns like curves, corners etc instead of big ass straight lines.

Processed inputs (ie all negatives brought to 0) are added back to the original input of the FFNN.

This helps the model retain the original signal (context from earlier layers), helps prevent the nw forgetting useful patterns, enables gradient flow through the networks.

Here is a GPT generated analogy on gradient flow, because everything I was coming up with it kept saying was wrong. “Think of a conveyor belt of chefs making a dish step by step. If the final dish tastes bad, someone needs to tell the first chef what went wrong. Gradient flow is like a clear message being passed from the food critic (the loss) all the way back to the first chef (early layers), so everyone can adjust how they do their part next time. If the message is too vague or weak along the way, the first chefs won’t know what to fix — and the dish stays bad.”

Remember at their core, NNs are stasticical machines, and a lot of stats is sifting through the data pile and making guesses, but guesses become estimations once we add a little reasoning, a little science and math to them.

Process output n number of times (where n is layer count).

In our case, it is 3 layers. Output from pass 1 is fed as input for pass 2, and pass 2 output is given for pass 3, and so on, but the process from step 5 - 7 is repeated in full for each pass.

FIN

Once we have processed the input through all N layers, were left with another sequence of vectors; one per input token. With this sequence, we can move to output generation. This is not really part of the transformer now, so we will assume the outputter is a magic box capable of giving me gold coins and a good nights rest.

So now we know (in theory) how transformers work. And we could (in theory) patch together our own implementation! But, much like upon finishing the hardware section of NAND2TETRIS, the reality is “you could, but noone does this because it already exists on the shelf”. This is not meant to be disliiusioning, in fact it is freeing. One COULD produce an entire CPU from nand gates, just as one could grab existing components and use them. It all depends what you want. I could use existing research and bookmark management software, or I could put my own together with string and glue. This is like the 4th or 5th time I have mentioned this project without posting it, I will get there.

DEMO

Our little demo, will now use the transformers package. There is a little bit of downloading we must do. Torch, transformers and the little model are approx half a gb. This will visualise the correlation that goes on during the attention mechanism. As we are not doing anything with the outputs as mentioned in step 11, there will be no follow through or response, because that is a very very different topic. Much as reading and writing are linked, they are taught independantly becuase after all, they ARE different topics.

pip install transformers torch matplotlib

Run this python script.

Each square in a heatmap represents how much one token (on the Y-axis) pays attention to another token (on the X-axis).

Brighter = more attention == more “relationship”

As we can see, shat has been broken up! This is because shat is not in its vocab, so it is being split up into “sub word units”.

What else is cool though is we can see my and cat, as well as cat and my are linked in heads 0 and 1 respectively. It is working!

Let us change shat to shit, and see what happens, since I have a feeling (well given I am writing this after conducting the experiment so I know) that shit should be in the list.

Check it out, theres a corrolation between cat and shit on head 1! And shit and on on head 3! Neat.

This concludes our cursory look into what a transformer is, and is another post Ive been meaning to make since like last October. I am getting better at scrubbing things off the to do list; I finally bought that nice new drill last night I have been meaning to get for ages.

Stay tuned for more halfbaked musings, hopefully the next one will finally be where I post about the workflows (bash and python loops) I have set up that literally make me feel like I have command over the palantir.

Patch diffing is an area I have been very interested in for a while; ‘patch Tuesday, exploit Wednesday’ from the days of old makes me incredible nostalgic for a time I didnt even experience, stolen valor and whatnot.

What follows is a primer on essentially getting your toolkit together and how to navigate pitfalls that I unfortunately did not avoid. This is a mashing of teachings from dozens of blogs and tuts; there is (until now) no single resource that spells all this shit out, as everything I have come across is the embodiment of pic rel

You know the incredible thing about memes is I dont even need to include the labels that usually live at the bottom of this picture for you to get it. I studied memetics at uni; it is an incredible fascinating topic, and it makes perfect sense that we are slowly moving to a post state where only the fragments of an idea are needed to convey the meaning, because it lives rent free in all our heads, so deeply embedded in our concisness.

TABLE OF CONTENTS

A TOC, this is new. Yeah this is going to be a long post. This is why I had to read so many different sources and why I am entitled to be glib about the process; theres a LOT you need to do before you even begin trying to trigger your first crash.

-Necessary Tools

-Acquiring Patches

-Setting up Ghidra

-Diffing with Ghidra

-Kernel Debugging with Debugger in VM via Network Debugging

-Kernel Debugging with Debugger on Host

-Verifying connection with windbg

Astute readers (and or those who sat my presentation) will note there is no section for actually writing the exploit. That was completley unintentional when I set down this path; I didnt realise that this (one collection of every stupid step that needs to be taken) doesnt actually exist yet.

Necessary Tools

-Ghidra Obviously we need this one. Ensure you get the latest version from the NSA repo. This is not a joke. Unzip somewhere you want to work from; there is no installer.

-BinDiff This does literally what the name says; it will diff two bins. This is pretty much the meat and potatoes of how patch diffing works; diff the patch against the non patched version.

Grab the compiled release, I see no reason to suffer through compiling it yourself; it comes from the Google repo, its as safe as Ghidra. I will make no further comments on this matter.

Install the msi. Take note of where it is installed, this information will come up later.

-BinDiffHelper As the name implies, this is a Ghidra extension to help BinDiff integrate with Ghidra. Unzip and place the contents into the “Plugins” folder where you unzipped Ghidra.

-BinExport Another Ghidra extension to further integrate what BinDiff is doing into Ghidra. Do the same as BinDiffHelper.

-Java Development Kit I used 21 for reasons I have forgotten. I cannot recall why and it is the middle of the night, I am not about to test with another version. Install it.

-ProcessExplorer Yes you read that right. Process Explorer. From sysinternals. Not procmon mind you, procexp. It is needed to dig yourself out of a pitfall you will encounter. Download it and put it somewhere, we dont need it yet.

Acquiring Patches

Updates can be found in c:\windows\software distribution\download

These will usually come in cabinent files.

use 7z and extract or expand -f:* UPDATE_FILE EXTRACT_LOCATION

You can use ./ to put it where you are if youve already got it in a folder

Patched versions of files live in c:\windows\winsxs

You can also find them at https://winbindex.m417z.com/

I have found this to be a very good source. I know nothing about the author and how reputable it is, but for the purposes of a short training demo and blog post, it was suffecient.

If you are wanting a higher degree of trust however, certainly use the files contained within the windows folder locations!

Now this is some insanely obvious advice on the surface, but I am ashamed to say it caught me out. If you have acquired a patch for say Windows 11, and you are looking to diff it against the unpatch file, it MUST be from the same OS! I cannot believe I wasted half a Saturday wondering what the flying fuck was going on when I tried to compare W10 and W11 files together.

BUT, it goes even more granular! Much like block upgrades to fighter planes, Windows subversions also are incompatable. W10 20H2 is a COMPLETLEY different beast to W10 22H2. You must compare not just similar blocks, but the SAME.

For the purposes of this exercise, we will be downloading tcpip.sys that I have uploaded. This is your prepatch, and this is your post patch. I have eloquently named them thusly before_11_24h2_tcpip.sys and after_11_24h2_tcpip Never fear, it matters not at all what you name them if they are the right files; the symbol server cares only for the hash, not the name.

Setting up Ghidra

Run the ghidraRun.bat file and when you are sternly told “NO ACTIVE PROJECT”, we will go ahead and make one.

File -> New Project -> Non-Shared Project -> Call it whatever you like and put it wherever you like -> Finish

File -> Install Extensions -> Green Plus -> Ghidra Plugin Dir -> BinDiffHelper Folder -> Ok

Repeat for BinExport

You will receive pic rel; a version mismatch. This is not a problem and it will work fine enough. Install anyway.

You will see BinExport in red.

Click OK, then OK again. Ghidra will not close when you ok the restart popup, you must do it yourself.

File -> Import, or press the i key to import files. We want both the pre patch and patched versions to be imported. The default options are all fine as they are, nothing needs to be changed for both.

Now when you import them, if you have a funny res like I do, and you are unable to see either the top or bottom of this big bastard window, we close it with esc.

Double click the files we have imported. You will receive a popup saying do we want to configure the plugins.

We say YES.

Tick the box in the resulting popup. Another popup will come up asking if we want to analyse the file. We say NO! We do not want to do this at this time. We havent set our symbol server up yet, so itll take an eternity to do nothing useful.

ALERT - ALERT - ALERT

This has happened to me 0 times so I do not know what the root cause is, but 50% of my training participants did not receive the configure plugin popup when opening the imported file. If this happens to you, we are going to (on the code browser window for the file we have opened) File -> Configure, click the little power plug and tick it here.

Back in the main project window, tools -> run tool -> version tracking.

Click the feet to start a new session.

Here is where we set the source and destination program. Now we need to have a little thinky here. We are looking for the differences between the unpatched version and the patched version. So we want to know whats different in the patched version. Which is our destination program; its where we want to go.

This is why it is very helpful to name your files appropriately so you can always identify with ease which one is pre and post patch!

Next, skip the checks, finish, configure plugins.

On each of the Version tracking windows that popup, we want to go File -> Load PDB File

We need to configure the symbol server now. Click the large red configure. It really shouldnt be red, surely another colour could have been used..

Set a local location for files to be downloaded to. I like to use C:\Symbols. Click the boxed icon to bring up the picker to select the folder.

Click the plus sign so we can specify alternative locations for Ghidra to look. You need this, because theres 0 chance of you having the necessary files ready to go, and like you setting a machine to use itself for DNS, wont know where to get info it doesnt have it in its own repo.

If it is all set it will look thusly:

To perform a search on the MS symbol server, you MUST click advanced EVERY TIME. I dont know why, but you have to. If you do not, it will only look inside the specified symbol folder and nowhere else.

We need to load the symbol file for both files. You can see here it doing a lot of stuff.

It might take a while. When it has finished (you will tell because the little blue bars stop moving) you MUST SAVE. After doing anything, you must save. Be mindful we are not actually doing anything against the real files themselves, but representations of little java objects who like to live in memory. And all these little java tools in their own little bits of java memory are all ephemeral until you SAVE THEM.

Once we have both files with their nice shiny symbol files, we are going to go back to the Version Tracking window and click the magic wand.

This will run a bunch of analysis over the two files. This is useful NOW, since we have applied the symbol files and Ghidra stands a chance of understanding what its looking at.

Once it is done, as with applying the symbol files, we need to save the Version Tracking window. Seriously just save all the time, everytime you do anything. Its all ephemeral until you tell it not to be, but you have to tell it not to be.

Diffing with Ghidra

Christ we are only halfway through this fucking blog. I knew I was avoiding writing this for a reason. Time for a coffee I think.

After running the checks, by default Ghidra prints all differences between the two files; stuff like variables, imports, headers. We are going to apply a filter to only see differences in functions. Remembering the objective here is to see what vulnerable function was patched out, we can see why this would be too noisy for us to sift through.

The lightbulb on right handside of the “Filter:” box is filter button. But theres two, because of course there is. The first one is the one we want.

There are lots of algos set by default. We untick all except •exact function bytes match •exact function instructions match •exact function mnemomics match •exact symbol name match

Untick data as the match type as well, leaving just function in the top left. It should look like pic rel. Press ok.

You might be wondering why we bothered installing all these plugins when Ghidra has its own inbuilt set of tools to perform these checks. The answer is simple; they suck. Look at this. This particular function which, spoiler alert, is the vulnerable function in question, is a completley different size in both files, yet still gets a 1.0 confidence rating on these being the same.

Enter our good friend BinDiff.

Go to before file, window, bindiff helper.

On the resulting page click the little wrench and tools icon.

You will note it lists the default bindiff install location. If this is where you installed bindiff, DO NOT BE FOOLED. It has not actually set the binary location! You still need to select the installed bindiff exe!! Default is c:/program files/bindiff/bin . I have no idea why this is the case, but if you do not set it, even if its where its default set, it wont work. V annoying.

Now remember the process: we are in the BEFORE file, wanting to diff it against the AFTER file. So click the open button next to the tools.

And select the after patch. Sanity check; the source is the before file and we want to select the after file.

Now we wait..until it hits 40%.

It will not progress any further. It cannot do it. There is something wrong with the way Ghidra calls BinDiff and it just cant do it. This is why we needed procexp.

Run process explorer, find the bindiff in there, double click it, copy the command line command, kill the process, open cmd and run the copied string.

If you do not kill the bindiff process before running it yourself, it will rpoduce a fucked file and be worthless and youll need to repeat this.

This will provide us a diff file in the specified output location. Mine tends to be in C:\Users\root\AppData\Local\Temp, but is specified in the copied command. If you are going to move the resulting vs file, you must move the other two files too!!

You will see the resulting error (lol).

Close the bindiff window, and click the new diff button on the source file again. This time we are going to select “Use externally created .BinDiff file”, and load up the VS file. You must click this obnoxiously small “load” button down here too. Next.

Now at the enxt screen, remember what we are doing. We are doing a diff of the source file against the changes made in the patch, as we have started the diff in the source file. The hash of the before will be green, indicating it is matching. Tick the radio button in first file and next.

We do NOT want to attach another file, we just go finish.

If you are seeing things like this, you have not applied a symbol file along the way. This is why we must save at every step. Close Ghidra, delete the bindiff files, reopen, reload the pdb, SAVE. Redo the wand, SAVE. Do the diff.

However, if you see output like so:

Then the diffing was succesful.

Note that Ipv6pProcessOptions is different. The cve is an ipv6 issue, so this is good.

If you do not have that particular function uptop, click on “Similarity”. This will sort the table, and is why it is important to ensure if you are following along, you update your tools! That functionality only came in during the time I was compiling this training session, and is a LIFE SAVER.

Using version tracker windows we can compare the decompiled code of both versions

The filter tab is actually a search bar. Silly name but anyway. Look for ipv6pprocessoptions. The filter tab is not case sensitive.

Once again, take note of the source length and dest length are different, yet the confidence and scores are still 1. It may have been a mission to get here, with these external tools, but they work and they work right.

Right click on the function, compare functions.

Scrolling is in sync view thank god.

Right at the end we can see there is a change made where IppSendError is called instead of IpSendErrorList after IppDiscardReceivedPackets is called

We can see iVar4 = Feature_552672569__private_IsEnabledDeviceUsage(); this is a particular ms function sometimes added to enable partial patch rollbacks.

It checks for a registry setting, which is set will make the function return false, so the original code is executed instead of the patched version. Because we know ms patches break things this lets a sysad “unpatch” a single vuln without removing the entire rollup

This functionality is not important for exploitation, but is helpful when diffing as it draws extra attention to what is being patched out. Remember we care about whats different between patched and unpatched, so we have now honed in on IppSendErrorList being the problem child.

Lets right click it and “Compare Matching Callees”.

The window will immediately switch to the new function; down the bottom of the Function Comparison window you can switch between the functions we have exploded.

So we can see the sum total of patch is for the machien to be using IppSendError instead of IppSendErrorList if the patch is set. Lets now see if we can hit any of these functions within a debugger.

Kernel Debugging with Debugger on Host via Comm Ports

Depending on how you are setting your environment up, ie with a virtualized debugger or putting it on your host machine, there are very different ways we connect to the target machine.

Now this is another one from the “well thats blindingly obvious” category, but it is worth mentioning; we are looking at kernel files here. We will be doing kernel debugging. How, can you hit breakpoints on the kernel on your vulnerable machine, if it is also holding the debugger? You cant, which begs the question of why its even an option in windbg and I will not be listening to answers because I do not care.

So we have two options; set windbg up in its own vm, seperate from the testing vm, or set it up on our host machine, also seperate to our testing vm.

First we will go through having windbg on your host machine. As proud as I am of the work I did to get mona running under windbg preview, I had little desire to go down the VM rabbit hole again (you will note there is a section of this post on that, as it turns out people in my training session did want that themselves)

And my faith was rewarded, as this is actually (comparatively) simple.

Configure the Target VM

You’ll need to set up a virtual serial port in VMware and configure Windows on the VM for kernel debugging. Step 1: Add a Virtual Serial Port in VMware

•Power Off the VM.

•In VMware Workstation/ESXi, go to VM Settings.

•Click Add Hardware → Select Serial Port.

•Choose “Output to Named Pipe”.

•Set the pipe name according to the rule set below

•Click Finish and then OK to save.

VERY IMPORTANT POINT HERE: this is another one from the Captain Obvious Sagas (TM) but it is worth pointing out; the VM MUST be set as the “server”. This means the machine is set to create the pipe, not just receive it. This will NOT work if you set the first drop down to client. The second one however, makes 0 difference which one you set it as. Both work and I have tested extensively.

Named Pipe Rules

MUST start with \\.\pipe\

This is a requirement for all named pipes in thr windows nt namespace

There are a LOT of rules regarding valid names, honestly best to sidestep all of them and just use word_word. slashes, colons, wildcards and certain single word names all have their own issues.

•underscores are fine

•az AZ 09 are fine

•tacs are fine

Linux does NOT have any requirements for named pipes, except that you can make the file. (since everything is a file on linux, ergo pipes are files). But thats not really relevant now, is it?

Enable Kernel Debugging on the Guest OS

Boot into the VM, pop cmd as an admin and run:

bcdedit /debug on bcdedit /dbgsettings serial debugport:1 baudrate:115200

I have spent time looking into the port numbering scheme. Workstation by default will assign the com port to port 0, within workstation. Windows however will take it as 1. This can be changed by modifying the vm ini files to assign another port, but I saw no need to dig into it. The point I am getting at is if you have added a single serial port, the port we want to set for debugging will ALWAYS be 1, despite what the hypervisor is calling it.

Restart the VM to apply changes.

Configure WinDbg on the Host

When trying to do kernel debugging, you MUST run windbg as an admin!

Once the debugee vm has booted, click file, and set up your settings as below. When possible, I like to break on connection. By default the port box will only contain a name, no suffixes. Prefixes? Do not be fooled by this; we must still include the .\pipe\ portion for the connection to work!

We are now ready to enter dbg; configuration has concluded. Click here to skip over the vm to vm configuration

Kernel Debugging with Debugger in VM via Network Debugging

There will be no screenshots here; I am at the end of my tether getting this together. VM to VM kernel debugging is a terrible way to do things, it is insanely slow between my vms for no good reason and everyone in the training session had the same experience. I will not ever be doing this, but it is noted because I want to purge these notes.

Find and install the Windows SDK. Why? Because Microsoft says so. I have read a LOT of MS doco in my sysad career, and very, VERY few times do they ever say “hey this is a bad idea”. Once was a fucked situation involving hybrid exchange server cutovers me and my hacking homie essentially developed the pathway for because the doco had no pathway, so that gives you some indication that when MS say “hmm are you sure” instead of letting you blow your own head smoov off as usual, you probably should pay attention.

Go to C:\Program Files (x86)\Windows Kits\10\Debuggers\x64, pop an admin CMD console kdnet.exe <HostComputerIPAddress> <YourDebugPort> .\kdnet.exe 172.16.250.56 50000 You can choose any port number from 49152 through 65535, 50000 is just the default when you open windbg.

You will receive a key, make sure to copy this or you will eseentially brick the machine (reversable, see a bit further below to fix)

Unlike VM to host where we want to connect once the target machines bootes, we set up the windbg vm so its waiting for a connection BEFORE restarting the target machine.

This is because kdnet is making changes to your boot config, where it will not continue to boot unless it makes the connection. So you need to be ready to catch the pipe its throwing at your stupid shiny melon.

Reboot the machine and wait. A LONG time. Dont freak out, it will most likely take about 15 minutes before making the connection to windbg. This, over dozens of reboots and trials and tests to prepare for the training session, seems to be normal. It will get there!

As we can see this is MUCH, MUCH slower than doing it via com ports, but if you insist on having a seperate vm, this is how one would acheive it.

To make your machine usable again we revert from snapshot (you did take one didnt you?) or force it to reboot a bunch by kiling the power, and when we get to the repairer, run bcdedit /set {default} debug off

You may notice this is different than what we ran to set it; the tldr is when not specified, bcdedits will amend current, which is not the same as default. We need to specify default because the current when in edit mode is {bootmgr} which does not actually use the debug option, so its amazing it even accepts it. it will NOT allow your machien to boot normally if you do not set it to edit the {default} at this menu!

Since we are not actually in the OS yet, to reboot run wpeutil reboot in the cmd that you can pop.

After disabling, to re set it up over winsock again we need to retun kdnet, doing so will give you your old key (as long as you do not change ports or hostname from previous time).

It is very advisable to NOT AT ALL set break on connection, it is VERY PAINFUL to try and get a booting windows to continue to boot.

Verifying connection with windbg

This is going to be a small little bit of dbg info, youd be better off finding another source for more indepth explainations of things but I will give you what I put out in the training session:

Once connected and the connection breaks (if it doesnt break itself but is connected, manually break with the big blue break) we run !process 0 0

You will receive output like thus

Where it lists the current processes. I like to use this as a final sanity check to confirm that yes everything is working as expected.

Like Ghidra, we want to set our symbol path.

.sympath srv*C:\Symbols*https://msdl.microsoft.com/download/symbols

The dir between the asterix is where on your debugging machine to store symbols. You can set it anywhere; I set it to use the same folder as Ghidra does. The second argument is where to get symbols from.

We must MUST now reload the symbols, with .reload

Pic rel will show that it is infact loading the symbol files.

We now verify that symbols are indeed loading:

lm v m tcpip lm to list module v to display verbose output m for module name filter

Once symbols are properly loaded, time to set breakpoints:

bp tcpip!Ipv6pProcessOptions

If the breakpoint is accepted, you’re now fully set up.

We can also set bp tcpip!IppSendErrorList to hone in on that particular function!

You can set breakpoints on any symbol that the debugger recognizes, as long as symbols are loaded properly.

Once the breakpoint is set, send g to get the target to go. Now we can hit it with a precanned poc available from here.

If you are unable to trigger a bsod when sending the vuln, it has been discovered that you must reduce the core count on your VM. Its actually astonishing this one got as much hype as it did; its basically unexploitable in the wild because you need VERY slow machines for this to trigger. Drop the core count, and ensure you have actually set an ipv6 address on the adapter you are wanting to hit (I used fd00::1 on the vm and fd00::2 on my attacking box just to keep things nice and simple).

This concludes the post. It has been ardous and onerous and I have been avoiding writing this for some time. I am glad it exists so I can quickly move through these small hurdles next time I go down the patch diffing route, but this is by far the largest post I have ever written and given my output over the past six months, I am a bit tired. As alluded to in the GRPO finetuning post, expect to see more gaming related content upon your next visit.

As per usual, this is a very delayed experiment that I meant to take off the to do list much, much earlier. I had pencilled it in for Q1 after unsloth dropped their article on GRPO fine tuning in limited VRAM environments, and its taken until almost halfway through Q2 for me to actually get around to it.

The whole saga has been a long story filled with sighs, as my new rig decided that it infact did NOT like the idea of being used, and my cpu developed thermal runaway issues. With replacement cpu in hand, and a crushing lack of progress on pretty much any of my goals for this year, I thought it would be good to take a break from my playthrough of pokemon crystal (with a wonderful clock hack so I can move it between day and night at will) and actually cross something off. Amusingly, much like this time last year, a few posts are going to go up in pretty much rapid succession.

WHY ARE WE HERE

I wanted to do a reasoning finetune on llama 3.1. I like it. It is, in terms of hours used, far and away my most utilised local model. It sits nicely on my nuc and every 2 hours crafts a terrible yugioh tweet (see posts here and blog post here) and powers the custom tools I have set up for article summarisation (come back later to see blog). The model works, I know it works, its been serving its many purposes extremley well since before Christmas, so I see no need to swap it out for anything else when it does what I need it to.

Fine tuning

Think of it like taking your bog standard civic and adding a nice new ECU; its still the same car under the hood, youre still using the same powerplant that came from the factory, but we have spent time and effort to move it in a particular direction with bits we can slap on.

now replace civic with llama, ECU with a reasoning LORA, and factory with Meta. This is finetuning.

the analogy is a bit stretched, but work with it.

Wait, what the fuck is a LORA?

Low Rank adaption. A “paramater efficient fine tuning technique”. Basically instead of doing a full training run on all the params, which in the case of LLMs is usually BILLIONS of them, LORAs instead add in freshly trained paramaters. How many params are injected isnt strictly “set” but you define variable r, which performs a bunch of nerd math to define how many params actually get tuned.

It is defined as follows: 2dr, where d == hidden size, r == value we set. ie 2x4096*8 = ~65k extra params per modified weight. So each lora modified weight is adding ~65k extra params, where a typical tune is to modify 2-4 weights per layer. For a 8b llama, theres 32 layers.

ergo weve got 32 layers, having 2 (as an example) weights being modified per layer (64 total) with ~65k extra params * 64 == ~4.16M params being modified ~0.052%

Fine tuning indeed, like finely grained sugar or some shit. The numbers speak for themselves, loras are ORDERS of magnitude more effecient to train than doing full training runs; a tenth of a percent of the effort required to train the whole model is expended. Consumed? Either way it makes a lot of sense.

Now we know a bit about loras, we can think of them as little phaser pedals we put into the the signal chain to effect/affect the guitar sound going through them. Take note here; it is PART of the chain, it does not get slapped onto the end like trying to route the cabinent through the pedal. It becomes part of the path.

Yes we started with a civic analogy because I was thinking about the first fast and furious movie but then i went and had dinner and watched a murder doco and ive forgotten where I was going with that analogy, so we have a new one now. It works because the pedal is a small, little piece that gets slapped onto the overarching music. It cannot replace the training the base model (guitarist) has sat through, infact the outputs are really dependant on the base being good. But it will allow things the base was not trained on; unless your guitarist is a terminator, theres a very good chance he cant actually make phaser noises with his les paul. Not even EVH could do that!

Reasoning

Can be added to a model through GRPO. Group Relative Policy Optimization. This is what produced the “thinking” tokens everyone shat themselves when deepseeks r1 was released doing it. Which is very interesting because I recall o1 doing it in June 2024 well before any other lab had put somethign out to do it.

GRPO is a pretty interesting RL technique; its not quite unsupervised, its not quite supervised, its a funny little middle ground. Rather than training the model on the ANSWERS in the dataset, we are going to train it to show its working out. Like how in math class youd get points for showing your working even if the answer was wrong, we want to condition the model to do this, to show working out. When it shows its reasoning, it gets a sweet, sweet reward. So we are basically perofrming pavlovian conditioning on the model.

“But I thought you said it wasnt unsupervised? This sounds awfully like your Street Fighter II Playing Neural Network” You are correct to be suspicious, but the difference is we arent just saying “figure out how to get the reward”, the rules associated with rewards are tightly controlled. Where as in normal supervised learning the bulk of prep goes into the dataset itself, with GRPO we apply most of our efforts into the ruleset the rewards come from. This is what makes it not quite supervised and not quite unsupervised, it is a bit of both.

“but whats the G for Group mean?”

Here is a 5 step process ripped from the unsloth GRPO blog on what is happening at a high level

1 The model generates groups of responses.
2 Each response is scored based on correctness or another metric created by some set reward function rather than an LLM reward model. 3 The average score of the group is computed. 4 Each response’s score is compared to the group average. 5 The model is reinforced to favor higher-scoring responses.

We can see here the “batching” of responses being the G and the averaging being the R for relative.

HOW TO

It is almost midnight and I am tired. I have two other blogs I need to start fleshing out, and honestly the actual training of the model was far less exciting than the what and how of GRPO. Here is a link to the article I followed containing no nonsense steps to get it working under Windows. The unsloth docs all use vllm, which is NOT supported on windows. Following this guide it took ~6 hours to get through the 350 training steps on the 8b model using a single 3060. Unsloth only supports single card setups and as such I was hamstrung from the get go. I seriously did not enjoy this project.

For the love of christ make sure you are using venvs. It will pull a LOT of shit down from the heavens. Here is the raw, unfiltered environment build steps. Noone will ever read this and I will never recommend anyone waste their time doing this, so I dont care to clean those steps up.

Results

Want to see some results? OK.

It added some reasoning (admittedly VERY BAD reasoning) but some none the less. Understandable, I barely scraped by in grade 12 maths and routinely received passing scores for reaching the wrong conclusion but demonstrating working, sometimes even enough from having the right answer and the wrong working out.

ENDING PARA

This project has been underwhelming, insanely underwhelming. Because I am capped to a single 12gb card, I have to train in quantized versions. So I am already gimped from the get go! To do a full epoch was going to take just over 220 hours; I am not burning 9 days of power just for this rinky dinky experiment, when all I can do is train a lobotomized model anyway! Just because I can take a retard and give him a thesaurus doesnt make him any less retarded. It is making me slightly sad I am simply priced out of doing any meaningful research in the LLM space, but, there are other avenues I can venture down.

It is somewhat interesting that you can just..step away from the bleeding edge, and come back in 6/12 months almost like the time you were away from the field didnt even matter; new SOTA will have arrived, new papers washing the old ones, on wards it goes. Maybe it is because my head hurts and I am sick, or I verbalised it last night at the pub with some mates, that all I can really do with my rig and research is make toys. If I want to do anything as good as or better than the big labs are doing, I need approx 100k of gear. Even then I think I am underestimating it by an order of magnitude, and it is just not worth it in the slightest when that could clear more than a quarter of my remaining mortgage.

At least it has washed me of any illusions that with a multi 4090 cluster in the garage, Id be able to produce knife fighting models capable of competing; even in the world where unsloth did support multi gpus (to not waste a metric fucktonne of my life waiting for tuning to complete) this 192gb cluster will only let me do full fine tuning on a 7b model. As neat as they are, LORAs are not equlivent to a full fine tune. Honestly I thought this would have been blindingly obvious to anyone who even partially looked into what they are, but apparantly people still go “do you have a study to back that up???” to even the most basic shit.

Anyway the point is I now know that theres zero, literally zero, point in getting another rig to stock with GPUs; even the local llama king with his monster fucking rig can only fully tune a 13b model. I can keep obsessing over stacking mac studios, lamenting project DIGITS, consider getting three phase power to run bulk 5090s, or I can pay $40 a month to OAI and move the fuck on with my life and use my box of scraps to simulate training ranges and shitpost

I get a lot of use out of that image. I am probably about due to rewatch mad men.

Astute resders will note i said i took “considerable” notes. Most of them did not make their way here, as a lot pertained to how to DIY an agentic framework from scratch, something I thought I wanted at the start of the year, but my infrequent OAI deep research useage shows that I infact, do not.

This post really doesnt serve much except getting these assorted notes out of my phone and into a format I can digest them later on again.

“Consistently, the most succesful implementations werent using complex frameworks or specialised libraries. Instead, they were building with simple, composable patterns”

Wow, so basically what I said here almost three years ago turned out to once again be correct? Who would have thought? Oh thats right, I did.

There is a delination that does need to be made; agents vs workflows. I am a big proponent of this image

Because all my little ML powered tools (which will be discussed soon I just havent gotten there yet) are quite simply little bash loops. Why? Because it works. Agents, by the strict definition set here (which huggingface also agrees with) have no place in what i want to acheive.

“Workflows are systems where LLMs and tools asre orchestrated through predefined code paths.”

“Agents are systems where LLMs dynamially direct their own proicesses an dtool usage, maintianing control over how they accomplish their tasks.”

I can smugly point to this because I am infact certified.

“When building applications with LLMs, we recommend finding the simplest solution possible, and only increasing complexity when needed. This might mean not building agentic systems at all.”

Look honestly this blog could end here. This right here IS the justification for why my shitty loops are perfectly fine; they are just advanced enough to do the thing they need to and no more.

“Workflows offer predictability and consistency for well defined tasks.” ie a well constructed bash loop

“Agents better when flecibility and model driven decision making are needed at scale.” ie utter insanity dreamed up by the utterly deranged

“If you do use a framework, ensure you understand the underlying code. Incorrect assumptions about whats under the hood are a common source of customer error.”

WOW so youre saying this shits JUST like programming? that the magic number machines are infact only as good as the operator?

“The autonomous nature of agents means higher costs, and the potential for compounding errors.”

“Success in the LLM space isnt about building the most sophisticated system. Its about building the right system for your needs. Start with simple prompts, optimize them with comprehensive evaluation, and add multi step agentic systems only when simpler solutions fall short.”

This, here, is the real reason for the loop meme. Its because 90% of problems are solved with the simpler solution. there are very, very few novel things on earth, and that includes issues. Almost every issue you can possible encounter has a been run into before, and therefore a simple solution can be crafted.

“Customer support and coding agents are [anthropics observed] two biggest success areas for agents”.

At face value, this reads like an endorsement for agents, but look at the inverse; they are saying ‘basically everything except customer support and coding were not succesful wrt agents’.

The final paragraph is from a secondary source I read at the same time. I dont remember where its from, or whom, and I dont care to look it up.

“Before we start building the agent, lets degine the goal of the agent

•Define problem

•Break problem into each part

•Build tooling for each part of the problem”

SHEESH. Thats been in my notes since JAN and it is MAY. I have been LAZY with my posting. I have approx half a dozen I have been meaning to get out the door, but just keep getting sidetracked and pulled into a dozen different little directions. It is interesting to observe the rotation of blog topics, from infosec to gaming to ML, with each of those three major topics each receiving different amounts of air time year on year.

Things at work feel kind of stable, and I dont feel the crushing death race to get ahead using AGI in the basement anymore. Twitter is extremley neurtoric and I will be ceasing to use it soon. 4chan is my real home, theres just nothing to be gained on twitter. Anyway the point I was driving towards is I think the ML topics are winding up for a little bit (I chuckle as the next 3 I have in draft are all ML related) and I have some interesting gaming things I want to discuss next.

Not until it dawned on me that the reason I can’t get behind the “"”new””” sensation of “agency” and “you can just do things” is they are all, every single one of them except me who is probably the smartest man on the internet right now, confusing choosing to do something, with being able to do something.

At first glance these both look the same. But this is where our good friend the synonym comes in. What is being able to choose something, but the freedom to be able to choose? So we can distill this as agency is being able to choose to do something, choosing to do something is freedom, therefore agency is freedom. Badda bing badda boom. QED and so on and so forth.

But this is meaningless in many ways. I have the freedom to choose to play in traffic or put my dick into a blender; so WHAT if I have the freedom to choose to do things? By virtue of being ALIVE I have the freedom to “just do things”. None of this moves the needle, and is nothing but a rediscovery of Tim Ferris’s thought experiments from the 4HWW. Asking “what would you do if you had 10x agency” means “what would you do if you had 10x more freedom”, and this is not at all the same thing as being able to do something.

The problem is the word “can”. You “can” just do things. You can choose to do things. You have freedom to do things. I have now belaboured the point enough that I am sick of writing about it.

The antidote is gumption. Which is your ABILITY to do things, not your FREEDOM to do things. Sure, I can CHOOSE, I can DECIDE to start manufacturing 3D fuckbots, but if I don’t apply the appropriate amount of gumption, this decision, this agency to choose is entirely and completley wasted. Who gives a shit if I can decide to do something if I don’t actually go and fucking do it?

And this is my core problem with the “cracked zigpot” Twitter sphere Thats slowly taking over my goddamn feed; all these cunts talking about choosing to do something, as if having the choice to decide to perhaps think about doing something is somehow analogous to actually going and doing something, and then producing three fifths of fuck all.

Because they lack gumption. Up and atom, drive whatever you want to call it (but not agency because it’s FUCKING NOT) is an entirely seperate thing.

It can be exemplified by a few tweets that I will list out in no particular order so I can remove them from my phone and instead reference this post next time I need to:

1)

2)

3)

Why are these important? Because all three of these exmplify actual agency; things being done. Look very closely at the first one. The sentiment of “You having a job == your startup is funded.” is such a novel but ultimately true expression of gumption. You receive money from working to afford the oppertunity to do different work.

None of this “omfg my life is pain im a staff/principal level and must quit being paid trumendous amounts of money in order to pursue my passion of putting slop into the world” type malaise i am seeing so fucking much of lately (my twitter feed is absolutley FUCKED) but actual, “this is HOW you do things”, none of this can bullshit.

If the token breadlines go down (this is such a good line, it really does sum up the dependancies we have on the oracle), I will simply utilise my small but mighty 3060s and raise my own palanatir. If this is not suffecient, I will acquire more of them until it is enough compute. Project digits is set to come soon, and the work being done to run monster models on M series mac is astonishing. This is truely, demonstrating agency. Not the “I have the freedom to choose to perhaps run my own models!”, but literally “yeah i bought this shit because i DO run them”, and in some cases such as the work getting the full deepseek r1 (not a distill) to run on what at the limit can be called a home computer.

“Why? Why should we fear to use it?” (if you replace ‘it’ with anything here; OSS, 3060s, r1 the sentiment stands. There is a LOT of takeaways here so use your noodle!)

Its also important to look at and dissect two things from tweet number 3; using what you have is effeciency, it is pragmatism, it is action and it is how things get done. using what you have to do what you can is essentially the tldr on how to make yourself a little life. I will not receive h100s from the heavens, and I have more important things to purchase with my hard won bennies than insanely expensive and very energy ineffecient GPUs; things such as removing my mortgage, getting my back tattooed and getting a neat little 3D printer.

So armed with my 2x 3060s (and if needed more of these little cheap sleepers), and the man armed with his 1070, we both intrinsincly know we are never going to run GPT4 level models. Hell even 3.5 is realistically out of the question. But being industrial and nimble little people, we can run SOTA models on very specific tasks. Which is how megazord is put together; a bunch of compotent little components joining together to make a larger droid.

This is a stretched analogy and I am tired so work with it a little. But perhaps this is the true way to get closed AI capabilities at home, by linking together experts. I am not talking about MOE the LLM architrcture, but a series of experts in a series of CICD pipelines, where different experts are called for each specific task. “mom can we have MOE” “we have moe at home” and its a series of python scripts.

A final thought that I am including just because. Let us be frank, cargo shorts and seedphrases have little to do with this concept; I just think its a neato thought. It is reminiscnat of the first bourne movie, where posessing nothing but a literal THOUGHT to retreive the contents of your vault you can use to springboard a new life. Admittedly yes he did need it embedded into his body to rememeber the phrase, but as the kids these days say, that is simply a “skill issue” and he should have upgraded his remembering game.

I was much angrier when I started this post. Then I was forced to write another, diffent angry post, and have lost some of my bluster for this particular topic. So I will conclude with a series of nice things that have occured in the past fortnight: I have received a large rate reduction on my loan, my pentests for a client I am sick of have wrapped up, I have finally gotten past that fucking annoying level on battle realms that has plagued me since the 5th grade (defending the swan pool), I have a series of compotent “"”agent””” (there is another post coming where I discuss agents) scripts cobbled together including pdf parsing and vision models, I found out a pretty neat new yugioh structure deck is dropping shortly, my new eink reader has arrived and my wife got me a very neat yeti in lime green.

Despite living on the stupid platform, I refuse to pay for twitter. No, it’s not happening. So I have been boxed into single thought tweets for my most recent argument, and in a sign that I somehow have won, have received no follow up replies. So instead I will detail my thoughts here and subject all you fictional readers to them.

After coming across this tweet morally grandstanding how open source tooling (I do NOT give one tiny iota if this is not the “OS” you people meant, you do not get to split hairs, fuck you, either demonize everything or nothing) I foolishly spent my lunch time arguing with two brick walls who want to start criminalising people who release GitHub repos, instead of actually updating my own repo with some agents.

The crux of it boils down to some people are upset that items they can freely select from the internet, can aid criminals. Do you know what else can aid criminals that they can obtain with extremely little cost (infact less than an internet connection?)

A hammer. I am sick to death of everyone saying “OH BUT AN OSS C2 IS LIKE GIVING EVERYONE A GUN!!!” No, its a TOOL, not a weapon. Please for the love of god use your noodle and compare redline stealer to psexec. “oh but people use psexec for all sorts of hacking!” Yeah and people use hammers to break peoples faces, we going to demonize hammers? Or just the people who use them in criminal ways?

Lets hammer (HUE) this point a little more. Cobalt Strike is used in attacks documented on DFIR report CONSTANTLY. a cracked, leaked version of the product. Why do I not hear the same thing from these people about how cobalt should be struck down from the ends of the earth? That Fortra should be held criminally liable? Is it because they think it is difficult to get a hold of? I am in 4 separate telegram chats that have zero entry requirements that have copies of it; if you want to find it, its out there and easily accessible. but no, the bad actors in this thought experiment (that is actually a real scenario) are those who cracked and leaked the product (Vlad and Ivan), not the creator (Mudge/Fortra).

So to extrapolate, those at fault for cobalt strike being used in real world attacks, are those who undertook criminal action? Shock and fucking horror! Yet when it comes to open source, half of cyber looks at it likes its open sores; with disdain and a desire to actively harm those who create.

“But thats taking it a bit far isnt it? People aren’t really saying that should happen!” They are, here is a follow up tweet to supplement the original. These people actively wish to remove livelihoods in order to clamp down on second and third order effects that may or may not result from some boogeyman in a 4th world shithole.

“Because somebody, somewhere, may possibly and potentially do something unkind to someone, I (they) wish to limit your career, cause irreparable harm to your ability to perform research, and require you to give up your skillset in order to drive a taxi or wrestle a crocodile because think of the children!” - completely deranged, braindead monkeys with the biggest hate boner for offensive who would rather see you unable to feed your family than actually apply the necessary effort to secure their environments.

It always comes down to this, people who wish you to stop doing a thing always come for your livelihood. “but others dont need to drop OSS to make a name for themselves!” How the fuck do you think Bobby Cooke (the genesis of this argument) even got noticed by IBMs xforce in the first place? By being a really friendly guy? Or was it perhaps his insane drops re bokuloader and his other cobalt work? hmmmm I have a feeling it was the second one!! I dont know about you, but I think companies like to see what value you can produce, rather than what you SAY you can do! just a sneaky suspicion!

“oh so you support hospitals getting randomwared?” the smug little avatar on twitter xeets, thinking he has caught you in a logical trap, not realising it is nothing but an appeal to emotion. The trolley problem does not work on me, because I will make it clear every time I will put the train on both tracks if it means my family stays safe. I’ve seen the stars and forks my Follina POC got; I know it is in bad peoples arsenals. Do you honestly think that thought gave Gaston Glock a single moments unrest?

pic rel, how i sleep knowing I have provided for my family using the means and hows I posess.

You know if you squint, theres half a political vein running in here. But thats not for today, it is not pertinent. The point is I am very sick of people placing blame on my trade and craft for the evils in the world. People ransomware things? That sucks, but your insistence that I should no longer feed my family as reparations for this crime I did not actually commit makes you just as evil as the one who ransomwared a hospital.

Alternatively titled “DIY self licking icecream cone”, “Spend a weekend building the worlds worst Pojos Forum clone”, and other very unclever titlings. tldr steps(ish) on how to overcome stumbling blocks I encountered when building out a very useless twitter bot that posts a Yu-Gi-Oh! card every two hours, complete with full card art from the ygoprodb and a description provided by an 8b param llama 3. Watch it be very wrong here.

It has dawned on me that I am actually entering another manic period, much like in June when I went through a number of projects in a period of weeks, I am doing the same thing again. It is like I have woken up from sleepwalking and am aware how much ground I have lost while trying to not kill myself, so I must produce and conquer projects at a rapid pace just to not move backwards. The list of items in my blog post list is increasing at an insane rate; armed with token generation machines and a fresh perspective I am the most powerful version of myself I could have ever imagined.

Imitation is the sincerest form of flattery or something to that effect I am lifting this project almost verbatim from my hacking homie, who had the idea of a GPT powered shitpost bot months before I did. However he has gone into the void and I dont know if he will return. As such, I have had to forge my own pathway to this project as his blog on it has not gone up.

The twitter API docs are missing a fuckload of info. Perhaps their postman collections were in better shape, I do not know because I chose to not look at them. It probably may have saved me effort, but who knows, theres a chance they suck too. This github repo helps with a lot of the issues I was having. As it says, tweepy does NOT handle the oauth2 flow, and my friendly robot overlord was pushing me to use tweepy at every turn.

So what is the problem we are trying to solve here (regarding oauth)?

Essentially we want one of these to popup, and we want to authenticate it, in order to receive a bunch of tokens we can use to post etc.

Sounds pretty simple on the surface, but I had to jump through a LOT of hoops to get to the end state because the docos seem to want to frustrate you and seem to dislike the idea of you choosing to perform this locally.

Here is the complete script I used to stand up the request and token capture server. There are amendments from the github repo; these are necessary for the way I handle refresh tokens (not the way I should, but it works). I will elaborate later on.

Obvs the port number listed in the script must reflect what is set in the twitter dev portal. It is no matter what you set, it just must match. Now this is the interesting part: the callback URI must be a complete URL. You canNOT just put in the callback URI, it must be the entire http://127.0.0.1:5000/callback if using my script. For the website URL section however, put literally anythign, as long as its a valid URL. I put mine down as the blog URL lol.

Remember: if you are developing a posting bot, you must give the app permissions to post.

Host the script, navigate to it, click the big ass URL, allow the app to manage the account, and scoop up all your tokens. We will be needing them.

You will note that the github example only collects the access token/bearer. Which is fine, if all you want to do is use the api for 2 hours. We need to also receive the refresh token, so we can grab new bearers.

YOU WILL ALSO NOTE THAT I HAVE ADDED A SCOPE THAT THE GITHUB WRITEUP DOES NOT. offline.access allows us to request the refresh token in the first instance. If we do not specify, and in turn authenticate with this scope listed, there will be NO granting of refresh tokens. Given this entire bots purpose is to sit on my rpi and shitpost autonomoslouly withOUT user interaction, we need the bearers to be able to be refreshed constantly.

So now we have authenticated, and you have some nice tokens you can use for shit. Your access token lasts 90 days, so you have some time to work out what the app is going to do.

Given the API documentation does not say theres anything against this, I decided the simplest way to deal with the bearers expiring was to simply request a new one every single time I post. I dont need to worry about the 2 hour expiration, because I dont care if its valid or not; it will be valid as I make the call for the updated bearer!

Now this is a sticky one that you should be aware of (its worth pointing out because I got stuck here) you are exchanging your refresh token for the bearer. Just because you have 90 days to use it, does not mean you can use it for 90 days. Important distinction.

We make a request to the token endpoint, sending off the current access token, and it returns a new bearer and a new refresh token. A two for one deal. Easy enough, just do this before every post post (huehuehue) and you will stay accessed.

Cool now we can post! But we can only post posts, we cannot upload images, because for some reason that functionality has not been ported over to the v2 api and does not support oauth2 authentication. Fucks sake.

Fortunately its actually pretty easy, we just generate api keys, access tokens and their secrets in the dev portal. These do NOT need to be autnehticated with a workflow like the v2 api keys, nor do they need rotating.

Two seperate steps need to occur; the image is uploaded to the server, and then the uploaded media is posted. It makes sense in a way, kind of, if you squint a bit. I have attached to demonstrate.

This is 1/3rd of the toy; yes it posts to twitter, but we need it to select a card out of the Yu-Gi-Oh! card database, and get a tweet from an LLM. I have attached here the scripts I used to pull the cards from the pre 5DS sets from the ygoprodb API, and one to pull the full card art. Theres not a lot to it, and no real reason why I got gpt to write them in powershell. I think I was in the middle of cloning virtual machines so I didnt have a linux box available. Next a little function is run that will randomely select one out of the CSV file I have of all pre-5DS cards, make a call to the DB to get full card art and saves it. I probably need to add a function to clean up the saved files before it tips my RPI over.

The irony is in all of this, the actual easiest part was the LLM. I have never heard of them before, and only stumbled upon them because I literally googled free llm api, and found groq. I like it, it looks like grugq. And they provide a fucking LOT for free, half a million daily tokens. I am going to use all of them in an upcoming project, but for now I am happy that this service exists for my shitposting bot.

Heres an example to send and retreive from the site. Its DEAD SIMPLE. You literally feed it your api key, what model you want to use and what your prompt is.

import os

from groq import Groq

client = Groq(
    api_key="APIKEY",
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Explain the importance of fast language models",
        }
    ],
    model="llama3-8b-8192",
)

print(chat_completion.choices[0].message.content)

Yeah I got fucking BANNED and all my api keys revoked because I was rotating keys to avoid rate limits. So now it all runs off my NUC, where a little 8b param llama 3 model is sitting doing cpu only inference. Fuck it, it works, I cant be banned from my own shit, and it powers my article summary bot. Which I havent posted about yet, but will..

So putting this all together, what do we end up with? Well our efforts reward with a shit poster. The space is intentional. I present the following post from last night that had me in absolute fucking fits of laughter at the absurdity of it.

This is another of those silly ideas that floated into my conciousness while on a flight a while ago, something dredged up out of my past desires I had long forgotten about. Originally I wanted to get an LCD hat for my rpi and have it display the card of the day on the hat (somehow I dont think I am using that word right) but given it lives connected to a USB port behind my television, I most certainly will not be looking at it.

Could this be done via an LLM on the rpi? It could, in theory. But these shit tweets are generated with am 80b param model; anything capable of fitting on an rpi would be an order of magnitude worse.

Here is the final script. THIS is the script post ban. Somehow, despite pumping project after project out, my to do list keeps fucking growing.