Tumgik
#implement all I've learned. :-)
gxlden-angels · 4 months
Text
My therapist and I have determined I have what I'm gonna call James Brain
It's basically a moral OCD subsection where it feels like every "bad" action is just as bad as any other "bad" action. Something like snapping when you're hangry feels like it's the same as robbing that person at gunpoint. It's based on the verse that's usually interpreted as all sins being equal:
"For whoever keeps the whole law and yet stumbles at just one point is guilty of breaking all of it." James 2:10
25 notes · View notes
thewatercolours · 2 months
Text
Podfic: "Captive Crown" by GerbilofTriumph
A shabby narration of GerbilofTriumph's excellent King's Quest fanfiction, "Captive Crown," complete with outrageous attempts at accents and enough bloopers to start a drinking game (with um, raisin juice. There are too many goofs for the real stuff.) This wonderful fiction, full of courage, nightmares, and healing, is gratefully recorded and shared with permission of the author, @gerbiloftriumph. Go check out her awesome creative blog.
All seven chapters are available at the link above, but if you just feel like listening to the first chapter while you scroll, voila:
Original text here:
11 notes · View notes
shinobicyrus · 1 year
Text
Cooking Things I Learned the Hard Way
Don’t use plastic utensils to prepare hot meals just. At all. The heat from the stove can soften and even melt the plastic, and then all that crap winds up in your food.
Do not microwave plastic tupperware containers with leftovers in them for the same reason. Put the food on a plate or in a bowl, first.
Don’t use metal utensils on your non-stick pots and pans, as it can scratch the non-stick coating and then start leaking that stuff into your food. Use wood or silicon utensils, instead.
Metal utensils are fine with stainless steel or cast iron pots and pans.
If you can afford it, I highly recommend investing in a good cast iron pan. They’re durable, get better with age, and can last you literal decades. Just don’t wash them with soap.
If you like rice, buy a rice cooker! There are some pretty simple, cheap, compact models out there that make rice perfectly every time. My roommate and I made it through quarantine with my little rice cooker.
Not technically cooking per se but try having a snack before you go grocery shopping. If you’re not hungry there’s less chance you’ll make impulse purchases and stick to your list - especially if you’re on a budget.
If multiple people are eating a meal, then one person should not be doing all the cooking by themselves! Even if you’re “not a good cook” you can chop or peel vegetables, fetch ingredients, wash dishes or clean counters as you go to make the after-meal clean up a little easier. Preparing meals is always more enjoyable when you do it together.
102 notes · View notes
queers4years · 1 year
Text
Fellas is it neurotypical to treat large social gatherings as a psychological simulation where you are both the scientist and subject 
24 notes · View notes
jinkis · 10 months
Text
i hate everything
Tumblr media
#about to go on a rant cause i'm  ✨ stressed  ✨#i'm currently at the very end of my masters degree so i can be a fully qualified secondary teacher#however due to the teacher shortage i have been allowed to start working as a full time teacher and have been doing so for the last year#i love my job i love the kids i love the staff members#what i dont love is having to finish off my masters degree when i've been working in the profession for a year#i have learnt more being in the classroom for a year than my whole 7 years at uni doing my undergrad and masters#i am TIRED of having to waste my time and come up with bullshit assessments just so i can get this stupid degree#schools are on winter holidays at the moment and i am meant to be relaxing or prepping for the term ahead#but no ... i've been stuck in my room for the past week doing uni assessments to prove that i know how to implement teaching strategies#and showcase how to write a lesson plan EVEN THOUGH NO TEACHER EVER WRITES ONE WHILE WORKING???????????#i just feel really bad for the kids cause i spend all my free time doing my uni shit or sleeping cause im so drained#i never have enough time to organise things for my lessons or come up with fun learning activities#usually i just do the bare minimum and pray for a pass#but this assessment just has a satisfactory/unsatisfactory mark so i cant just half ass this shit#also can yall believe that im only half way through this assessment??????????#i still have so much more to do ughhhh#anyway if you read all that im sorry and thank you#marie.txt
2 notes · View notes
antmimicry · 1 year
Text
I love xkit's cleanfeed but it does sort of suck that I can't see the alt text caption (also enabled via xkit) until I hover over (and therefore reveal) the image itself. Feel like this could be fixed very easily with some simple css/js but I can't be bothered to do it rn ...
2 notes · View notes
squishy-min-mochi · 9 months
Text
It’s important to recognise that Barbie (2023) criticises both the patriarchy AND the matriarchy. Yes, the Ken’s are just accessories to the Barbies. Yes, they don’t have any say in the government they live under. That’s the point, you’re supposed to feel awful, you’re supposed to want the Kens to have their own agency, you’re supposed to want equality. The Barbie movie explicitly states that the way Barbie treats Ken is wrong, so much so that once he finds a safe space for his masculinity and individual identity he’s so excited to share it with the other Kens.
But they go overboard and replace a matriarchy with a patriarchy and now the same issue exists but in reverse. That’s the POINT!! THATS THE POINT!!! Barbie is not anti-men it’s pro equality PLEASE understand this
13th Aug 2023 UPDATE:
Heeeeey howdy!!
Due to the IMMENSE comments and discussion on this post (thanks ya’ll!!) I’ve decided to update my post with my recent opinions and hopefully clearer explanations!!
First, my original post only considers a very small and very vague analysis of the film!!
Since making this I've read all your comments and learned quite a bit about the matriarchy as it appears in human civilisation. Originally, I was pitting the patriarchy and the matriarchy against each other as though the results of their implementation were equal in the film.
They were not!! Below is the definition of matriarchy I’ll be working off of.
Matriarchy Simple Definition;
Matriarchy is a social system in which women hold the primary power positions in roles of authority. In a broader sense it can also extend to moral authority, social privilege and control of property.
There's a lot to talk about in the Barbie film that would fit better in an essay, so I'll try and condense it into this;
To me, Barbie (2023) is a film about the female experience and the shared connection between women that persists through childhood and adulthood, support and harassment, suffering and joy, mother and daughter.
It uses Barbie as its figurehead because of the immense societal and political impact the doll has had on women, both good and bad (as explained in the film).
The male experience as seen in Barbie (2023) is not the sole focus of the film- rather, it's an accessory (as the Kens are) to Barbie's story, and a necessary aspect of exploration to truly highlight the importance of individualism and healthy personal exploration.
I want to make clear that I in no way think the treatment of the Kens was just as bad as the treatment of the Barbies. I also still agree that the matriarchy fostered by the Barbies wasn’t good for the Kens.
Additionally, I’m aware that this take on Barbie (2023) works strictly within the assumed heteronormative boundaries of gender. There is a lot of nuance in the Barbie film and I don’t think everything can be covered or explained in on Tumblr post— but I hope this clarification helps!!
I hope you're all coming to your own conclusions and analysis of the film in a way that makes sense to you. And for those of you engaging in online conversations and discourse about it, I hope you're keeping yourself and others happy and safe!!!
Much love to you all!! < 3
65K notes · View notes
janmisali · 1 year
Note
what do you think of tone indicators in general?
unfortunately my thoughts on tone indicators are somewhat nuanced. fortunately, this is tumblr not twitter, so I can just write out my full thoughts in one post and be as verbose about it as feels necessary.
speaking as an autistic person (and I know there are other autistic people who don't hold this same view, this is just my perspective), I think as an accessibility tool, the extended set tone indicators in current popular use is fundamentally misguided.
the oldest ones, /s for sarcasm and /j for jokes, make sense. their notation isn't the most intuitive thing ("does /s mean sarcastic or serious?") but it's not too difficult to explain what they mean. I've had to spend my whole life learning by brute force what different tones of voice mean and what they change about how I'm supposed to interpret something, so I already know what "read this in a sarcastic voice" and "read this as a joke" are supposed to mean. my existing skills can be translated into the new form without too much effort.
the same thing applies to emoji and emoticons. I know what facial expressions mean, because I had to learn what they mean. figuring out if :) is sincere or not from context is a skill I've already needed to develop. it doesn't come naturally for me, but it's something I already at least somewhat know how to do.
most of the tone indicators in current use uh. don't work like this.
tone indicators like /ref or /nbh don't correspond to specific tones of voice. I don't have a "I'm making a reference" voice or a "I'm not talking about a person who's here" voice that I can picture the sentence being read in. these do not indicate tones, they're purely disambiguators. they clarify what something means without necessarily changing how it would be read out loud.
and on paper, that's fine, right? like, it's theoretically a good thing to take an otherwise ambiguous statement and add something to it that clarifies what you meant by it. the problem is that these non-tone tone indicators are not even remotely self-explanatory. it's up to me, the person who is being clarified to, to know what all these acronyms are supposed to mean, and how they change the way I'm supposed to interpret what something means.
it's, quite literally, a newly-invented second set of social cues that I'm expected to learn separately from the set that I've already spent my whole life figuring out, and it works completely differently.
sure, these rules are (in principle) less arbitrary than the rules of facial expressions and tones of voice and how long you're supposed to wait before it's your turn to speak, but they're also fully artificial and recently invented, which means they're currently in a constant state of flux. tone indicators go in and out of fashion all the time, and the "comprehensive lists" are never helpful.
in theory, I appreciate the idea of people going out of their way to clarify what they mean by potentially ambiguous things they post online. if it worked, that would be a really nice thing to do.
however, sometimes I imagine what the internet would be like without them. what if instead of using /s, the expectation was that if you're sarcastic online there's no guarantee that strangers reading your post will know what you meant? what if instead of inventing more and more acronyms to cover every possible potentially confusing situation, we just... expected one another to speak less ambiguously in the first place?
so, I on paper like the idea of tone indicators. I think it's good that some people are trying to be considerate by being extra clear about what they mean by things. but if tone indicators didn't exist, and people who wanted to be considerate in this way instead just made a point of phrasing things more clearly to begin with, I think that would be vastly preferable to even the most well-implemented tone indicator system.
also /pos sucks because there's something deeply and profoundly wrong for an abbreviation that means "I don't mean this as an insult, don't worry" to be spelled the same way as an acronym that's an insult
7K notes · View notes
azuremist · 8 months
Text
A guide to all accessibility-related needs on the indie web!
Tumblr media
I've been sharing this site around in all sorts of places, but I wanted to post it here, too - the Accessible Net Directory!
The indie web has a problem with accessibility. Namely, it seems like very few creators care to implement it. Which I get! It seems daunting to tackle at first. But also, it makes the indie web really, really difficult to traverse for disabled folks.
The Accessible Net Directory is two-fold in its uses.
1) It provides disabled people a list of safe indie sites
In order for a site to be featured on the directory, it needs to meet basic accessibility guidelines. This includes (but isn't limited to) no autoplay, unlabeled flashing images, low contrast, small / hard-to-read fonts, keyboard inaccessibility or non-described images.
Additional, non-required accessibility options for the directory are encouraged, and listed in a site's description on the directory. (Such as zoom-friendliness and different font options.)
2) It provides webmasters a ton of resources for implementing accessibility
It not only provides a list of steps one can take to make their site more accessible, but also provides the "why" of each step.
For example - one listed guideline for the directory is to use rem/em instead of px units for font sizes. Why? Well, the directory explains, "Users with low vision may need to increase the size of the text in order to be able to read and see better. Therefore it is important to always use scalable units for text."
Then, it goes on to list RESOURCES for changing px to rem/em!!
This site is an absolute treasure trove of resources for making your site accessible, and doing so with ease! I've used the resources listed to make my own site more accessible, and it's made the process seem so much more manageable!
So! If you run a personal site, then give the site a look, learn about accessibility practices, and apply them to your site! An inaccessible web is not a free web!
2K notes · View notes
d0youc0py · 6 months
Note
So, i've read through a lot of your stuff, and its giving me instant serotonin, and i already love your writing, I also have a rather interesting ask, and you can completely ignore it if you wish to, but!- TF141 + Alejandro and Rudy learning that the new addition to their team is a bit younger than them(around 17-19), and a bouncy ball of ditzy airheaded-ness, and actually sees a them as Brother/Father figures? A little bit of a comfort for those, and myself included, who have platonic attachments to these badass men!
Tumblr media Tumblr media
Your relationship with Price comes the most natural. He has always had an overwhelming desire to nurture and mentor those who are close to him- or those that he believes could become close to him. Considering he recruited you, he definitely saw you as one of those lucky few. You are a bit of a mess, needing some serious fine tuning, but you have an obvious want to learn and you soak up every bit of information he shares with you. He sees a lot of himself in you. He was only 16 when he joined the military and he wished he had a mentor to guide him and teach him so he didn’t have to learn from his mistakes, (not that he would’ve listened).
He was also the gateway to getting the other TF members to trust you- especially Ghost.
There is a lot of uncertainty in your job, but one thing you can always count on is that Caps got your back. You need a place to crash on deployment? He has a guest room. You need life advice? He feels like he hasn’t done anything right- but he’s been through enough things to know a thing or two. You had a nightmare? He office door is always open.
Tumblr media
He’s a bit over meeting new people. And he lets you know that. For a long time he saw you as a reckless pain in the butt- a risk. You were a powerhouse on the field, yet you lacked that certain refinement. It wasn’t till he saw you and Price during a training. The Captain had your full attention and he saw an eagerness to learn that he hadn’t seen for a long time. He decided to give you a shot, throwing out tidbits of wisdom to see how you would respond. It would always surprise him when you not only listened but implemented what he had told you.
It was like a switch had flipped in his brain and he became the older brother that he had always so desperately tried to push down. He knows all your little quirks. What gets you fired up. What calms you down. He’s incredibly observant and it feels like he knows what you need before you do.
“Keep sleeping, I’ll cover your watch.”
“If you forget your MRE one more time.” He huffed taking out the said MRE you forgot out of his bag.
He’s the definition of tough love. His toughness keeps you grounded and on track. His loving side makes sure you are always taken care of (even though he wants to strangle you daily) and the first time you introduce him as “your brother” he nearly had to excuse himself to dry his eyes.
Tumblr media
You’d think that the two of you would be partners in crime. Running around, causing trouble, giving everyone a headache. While the two of you do have your moments, Soap understands his responsibility as not only the elder, but the ranking officer in your relationship. Outside of the work the two of you are as thick as thieves, but out on the field it a whole different story. He’s serious, taking his self appointed job as your mentor to heart. This is the first time he’s taken someone under his wing and he is not going to screw it up. He teaches you as much as he can, as much as you need to not only be safe but successful.
The change is a shock to the TF at first, but they admire the way Soap “ages up” and it inspires them to do the same.
Tumblr media
He doesn’t treat himself of your “mentor” or “elder” he treat you as though you two are equals. He treats you with respect and never talks down to you and he makes sure everyone else does the same.
The two of you hit it off instantly. Snarky comments are constantly flying back and forth between the comms and despite their better judgment the rest of the TF can just fight back a chuckle. He teases you relentlessly (out of love). All you have to do is make one slip up and this man will never let you forget it. But he also makes sure that if he can dish it out, he can take it too. This results in the two of you laughing hysterically, doing your best impersonations of each other. After long missions the two of you can often be found on the couch fighting over a blanket, watching your favorite comfort movies. You really are the sibling he never knew he needed.
Tumblr media
Like Price he sees a lot of himself in you. You have drive. You do have a tendency to be a little too aloof for your own good, but it’s nothing he can’t handle. He embraced you with open arms and makes you feel like you’ve been family for decades. He’s hard on you and demanding, but it’s just because he knows what you are capable of. He understands the trust you place in him and will do everything in his power to live up to the image you have of him in your head.
You have a permanent room in his house. He always makes sure to have the ingredients to your favorite dish on hand just in case you had a hard day. He makes sure all of your trainings are up to date so you have every opportunity to make it out alive. You are so important to him and he never lets you forget it.
Tumblr media
He’s wary of you at first. Even after Alejandro welcomed you with open arms. He saw you as a flight risk and he didn’t want to end up dead because of it. It wasn’t until one night you confessed how much you looked up to him- how you wanted to be like him.
He had misjudged you, and he apologized for it. He made up for lost time by going out of his way to be extra kind to you until it just turned into a habit. You and Alejandro both wear him out, he wouldn’t trade either of you for the world. He’s the voice of reason when you get ahead of yourself and he’s always the one who slows down and explains things to you when you find yourself lost. His patience seems almost endless and you thank his everyday for it.
Thank you for your kind words! Hope you liked this.
593 notes · View notes
spocks-kaathyra · 8 months
Text
thoughts about the Cardassian writing system
I've thinking about the Cardassian script as shown on screen and in beta canon and such and like. Is it just me or would it be very difficult to write by hand?? Like.
Tumblr media
I traced some of this image for a recent drawing I did and like. The varying line thicknesses?? The little rectangular holes?? It's not at all intuitive to write by hand. Even if you imagine, like, a different writing implement—I suppose a chisel-tip pen would work better—it still seems like it wasn't meant to be handwritten. Which has a few possible explanations.
Like, maybe it's just a fancy font for computers, and handwritten text looks a little different. Times New Roman isn't very easily written by hand either, right? Maybe the line thickness differences are just decorative, and it's totally possible to convey the same orthographic information with the two line thicknesses of a chisel-tip pen, or with no variation in line thickness at all.
A more interesting explanation, though, and the one I thought of first, is that this writing system was never designed to be handwritten. This is a writing system developed in Cardassia's digital age. Maybe the original Cardassian script didn’t digitize well, so they invented a new one specifically for digital use? Like, when they invented coding, they realized that their writing system didn’t work very well for that purpose. I know next to nothing about coding, but I cannot imagine doing it using Chinese characters. So maybe they came up with a new writing system that worked well for that purpose, and when computer use became widespread, they stuck with it. 
Or maybe the script was invented for political reasons! Maybe Cardassia was already fairly technologically advanced when the Cardassian Union was formed, and, to reinforce a cohesive national identity, they developed a new standardized national writing system. Like, y'know, the First Emperor of Qin standardizing hanzi when he unified China, or that Korean king inventing hangul. Except that at this point in Cardassian history, all official records were digital and typing was a lot more common than handwriting, so the new script was designed to be typed and not written. Of course, this reform would be slower to reach the more rural parts of Cardassia, and even in a technologically advanced society, there are people who don't have access to that technology. But I imagine the government would be big on infrastructure and education, and would make sure all good Cardassian citizens become literate. And old regional scripts would stop being taught in schools and be phased out of digital use and all the kids would grow up learning the digital script.
Which is good for the totalitarian government! Imagine you can only write digitally. On computers. That the government can monitor. If you, like, write a physical letter and send it to someone, then it's possible for the contents to stay totally private. But if you send an email, it can be very easily intercepted. Especially if the government is controlling which computers can be manufactured and sold, and what software is in widespread use, etc. 
AND. Historical documents are now only readable for scholars. Remember that Korean king that invented hangul? Before him, Korea used to use Chinese characters too. And don't get me wrong, hangul is a genius writing system! It fits the Korean language so much better than Chinese characters did! It increased literacy at incredible rates! But by switching writing systems, they broke that historical link. The average literate Chinese person can read texts that are thousands of years old. The average literate Korean person can't. They'd have to specifically study that field, learn a whole new writing system. So with the new generation of Cardassian youths unable to read historical texts, it's much easier for the government to revise history. The primary source documents are in a script that most people can't read. You just trust the translation they teach you in school. In ASIT it's literally a crucial plot point that the Cardassian government revised history! Wouldn't it make it soooo much easier for them if only very few people can actually read the historical accounts of what happened.
I guess I am thinking of this like Chinese characters. Like, all the different Chinese "dialects" being written with hanzi, even though otherwise they could barely be considered the same language. And even non-Sinitic languages that historically adopted hanzi, like Japanese and Korean and Vietnamese. Which worked because hanzi is a logography—it encodes meaning, not sound, so the same word in different languages can be written the same. It didn’t work well! Nowadays, Japanese has made significant modifications and Korean has invented a new writing system entirely and Vietnamese has adapted a different foreign writing system, because while hanzi could write their languages, it didn’t do a very good job at it. But the Cardassian government probably cares more about assimilation and national unity than making things easier for speakers of minority languages. So, Cardassia used to have different cultures with different languages, like the Hebitians, and maybe instead of the Union forcing everyone to start speaking the same language, they just made everyone use the same writing system. Though that does seem less likely than them enforcing a standard language like the Federation does. Maybe they enforce a standard language, and invent the new writing system to increase literacy for people who are newly learning it.
And I can imagine it being a kind of purely digital language for some people? Like if you’re living on a colonized planet lightyears away from Cardassia Prime and you never have to speak Cardassian, but your computer’s interface is in Cardassian and if you go online then everyone there uses Cardassian. Like people irl who participate in the anglophone internet but don’t really use English in person because they don’t live in an anglophone country. Except if English were a logographic writing system that you could use to write your own language. And you can’t handwrite it, if for whatever reason you wanted to. Almost a similar idea to a liturgical language? Like, it’s only used in specific contexts and not really in daily life. In daily life you’d still speak your own language, and maybe even handwrite it when needed. I think old writing systems would survive even closer to the imperial core (does it make sense to call it that?), though the government would discourage it. I imagine there’d be a revival movement after the Fire, not only because of the cultural shift away from the old totalitarian Cardassia, but because people realize the importance of having a written communication system that doesn’t rely on everyone having a padd and electricity and wifi.
679 notes · View notes
ao3commentoftheday · 1 year
Text
Since there are a lot of new people on tumblr these days, I'm going to start this whole thing off by saying that this is my personal blog and while this blog does have AO3 in the name, this is in no way official or affiliated with whatever OTW might have to say on this subject.
Yes, I've seen the reddit post  about the GPT-3 bot scraping AO3. Yes, I'm aware that Sudowrite.com are using the data from that bot to generate text.
A few things I've learned as I've looked into this:
1. Bot scraping is legal. If a website is publicly available on the web (does not require a user to login in order to see its contents), then they don't have grounds to try to stop a bot from doing what anyone can do. Here's an article by the Electronic Frontier Foundation about why this is the case  and also why it could be considered (on balance) a good thing . For example, scraping websites helps academics and journalists do their work.
2. Elon Musk doesn't own GPT-3. He's listed as one of the founders of OpenAI, the group who created GPT-3, but he resigned in 2018. He could still be a donor, but he has no official capacity in the organization.
3. Sudowrites is a tool that generates text, but it is a writing assistant not an AI author. It can not structure a story and develop a plot independently. It can not do research. It is meant to assist a human author by giving them prompts or ideas, helping them find a word or a phrase. But anything created solely by the bot would be at least somewhat incoherent and also in danger of committing plagiarism. For more information, I recommend this article.
What does this all mean? First of all, just because it's legal doesn't mean you have to like it. I'm not a fan of it, myself. but I also know that Google scrapes AO3 in order to provide search results for fans trying to find fic so I've kind of resigned myself to it.
Second of all, there's nothing AO3 or the OTW can do about it, really. There's a technical fix they can implement to prevent scraping by one particular bot (the one mentioned in the reddit article), but that's about it.
You, as an author on AO3, could lock your works to the Archive (restrict access to only logged in users). This might or might not protect your works from scraping. I don't know enough about these bots to give you an answer one way or the other. This feels gross. I understand that. I feel it too. Do what you need to do to feel better.
The original reddit post author states that they contacted the OTW Board, so there's no need for you to write in to AO3 Support. They're already aware of the situation.
3K notes · View notes
sparklingself · 1 year
Text
𝐭𝐞𝐧𝐝 𝐭𝐨 𝐲𝐨𝐮𝐫 𝐢𝐧𝐧𝐞𝐫 𝐠𝐚𝐫𝐝𝐞𝐧
Tumblr media
do you know yourself? do you know your imagination? the most important job for you is to learn to know your imagination. how can you do that? by spending time in it, by observing, by testing.
there are some fundamental questions you must ask yourself if you want to apply the law in a successful way.
does imagination actually create reality?
is imagination enough to fulfill my goals?
can i honestly imagine something i desire and it comes to pass in this world?
etc.
i will answer yes to all of these. but will you? how do you know for sure if you don't test it? the only reason i follow the law is because i have empirical proof it works. i've proved to myself that my imagination creates and i do not need to raise a finger in order for it come about.
the law is all about inner-work. no matter how many blog posts you read or how many of neville's lectures you listen to, the thing that matters the most is that you learn to be within yourself. that's how you master the law.
"Now we will go back to the 2nd of Genesis. It is said "And God placed man in the garden of Eden to dress it and to keep it." Now when you read the story you think it happened thousands of years ago. I have come to tell you it is now. You are now in the garden of Eden and you think you are shut out or banished. You are in it, and the garden is your mind, but you need--like every gardener--you need pruning shears. For you have slept, as you are told in that second chapter; having slept, weeds have appeared in the garden and the weeds are revealing themselves by the conditions and the circumstances of life. For your garden is always projecting itself on the screen of space, and you can see by looking carefully at your world what you allow to grow in the garden of God. But you have a mission, you have a purpose, it is not to amass a fortune--you can do it if you want to--it's not to be famous, it is not to be some mighty power, but simply to tend the garden of God. That's your purpose. You are placed in the garden to dress it and to keep it, that only the lovely things grow in the garden of God ." - Neville
you need to learn to tend to your garden. every day you need to water the plants and pull out the weeds in order to keep it flourishing. so create a routine, create habits that let you explore your inner-world. that could be journalling, meditating, etc.
edward art proposed a ritual in one of his posts you may want to implement: every day in front of the mirror when you get dressed be aware of your thoughts. are you feeling and thinking the thoughts you desire or are you self-sabotaging? make your thoughts lovely and feel the love.
little rituals like this really make a difference. curate something for yourself that fits into your day. it doesn't have to take a lot of time and energy. it just needs to be enough to keep you focused. the point of these rituals and routines is for you to meet your objective in your mind. all things are possible to the imagination and it will rearrange the outside for you.
1K notes · View notes
Text
i know i've said this before but i'm so tired of people acting like Patton is the general embodiment of morality. as if he is the personification of the legal justice system and everything he does must be good and fair™.
folks, Patton is Thomas's morality. remember what Virgil said in the Q&A episode?
Tumblr media Tumblr media
this applies to all of the sides. every single one of them.
Remus wouldn't be the same for everyone, different people have different types of intrusive thoughts. Logan wouldn't be the same for everyone, different people have had different levels of education, different forms of intelligence and different applications of logic.
but the funny thing is, i never see people criticize the other sides for being too representative of their title. they don't reprimand Virgil for making Thomas anxious because that's his job. they don't bash Roman for being a perfectionist. but for some reason, people expect Patton to be a saint who does no wrong.
Tumblr media Tumblr media
it's emphasized so clearly in multiple episodes that the sides are the result of Thomas's upbringing specifically. Patton is simply the moral code that Thomas's parents and on a larger scale, society, had taught him.
“Patton was so accepting of Virgil but he couldn't accept Remus or Janus!”
yeah. because it's easier to deal with anxiety than it is to deal with intrusive thoughts. don't get me wrong, anxiety is definitely a serious issue. but Thomas's anxiety never went directly against his morals, if anything, Virgil's perspective aligned very much with Patton's. being anxious about say embarrassing yourself in public or not getting your work done in time is not as morally concerning as having thoughts about murdering your brother.
but Thomas, like many of us, grew up with no one to tell him what intrusive thoughts are or how to deal with them. when Patton was chastising Thomas for having these thoughts, that's just Thomas beating himself up over something he can't control.
the same goes for Janus. Patton had a hard time accepting Janus because Thomas had learned to always be selfless since when he was younger, and Janus was challenging that point of view. when Patton was pushing Thomas to be selfless, that's just Thomas pushing himself and feeling guilty at the thought of focusing on his own needs.
i feel like people think that Patton is actually Thomas's father figure or something. i understand if he reminds you of someone toxic in your life and it can be uncomfortable to watch. but while it's good to relate to the media you are consuming, it's also important to view things in context.
Patton is not a person, he's not an individual who has the freewill to do what he wants. he is only Thomas's morality and what his morals are heavily depends on what Thomas was taught and how he was raised.
Patton cannot control the morals that Thomas had learned. he can only try to implement them in Thomas's life and unless Thomas himself decides to unlearn some of the unhealthy ideals that he has internalized, Patton cannot go against his nature. he can only do what he has been programmed to do, with the knowledge that he has.
Tumblr media
323 notes · View notes
glassbirdfeather · 3 months
Text
Mohg's Brain
(This is an essay on Mohg, Lord of Blood, from hit video game Elden Ring. It just takes a bit to get there.)
Tumblr media Tumblr media
There is a story often repeated in Psychology classes, Physiology classes, pop psych media like YouTube, podcasts, and garbage daytime television on channels that used to be scientifically rigorous: about a man with an incredible brain injury. For those of you who haven't heard the story or are not yet sick of hearing it, I've included it from memory below, because I have heard it just that many times.
If you've heard this story already, you can skip to the subtitle: "Can We Even Learn Anything From Gage?"
If you already know the controversies about Phineas Gage or just want to jump to the part about the video game character, you can skip to the subtitle: "Let's FINALLY talk about Video Game"
----
"The Curious Case of Phineas Gage"
Tumblr media
Phineas Gage was a railroad worker who would help clear land with explosives. The dubious and definitely wouldn't-have-been-OSHA-approved method of laying these explosives was to chip a hole into the mountainside, place the explosives, and then tamp it down using some sort of implement like a railroad spike. What happened next was predictable and it's surprising this didn't happen much more often--when packing the explosives, they detonated in Gage's face. Specifically, this launched the spike underneath his left eye and out of the top of his head. Less predictably, Phineas stood up afterwards. When a doctor arrived, said doctor did not believe what had occurred until Gage vomited approximately a "teacupful of brain matter" onto the street.
Due to lack of effective sterilization and antibiotics at the time, poor Phineas Gage was bedridden for several months, where he continued to lose further brain matter to infection. Eventually, he did recover, although he would continue to experience migraines and seizures for the rest of his life. While he lost his job for the railroad service, he went on to work in a sideshow attraction, carrying around the very railroad spike that went through his head. Eventually, he got a job and worked as a taxi driver and lived for several more years before dying of a seizure.
Phineas Gage was never the same after this life-altering injury: he was belligerent, drunk, lied frequently, and lost his job for the railroad company because of his new personality. And I do say NEW personality--Phineas had become like a completely different person and was, in essence, "no longer Gage" (they love quoting that). The damage to regions of the prefrontal cortex made him unable to make moral judgements, and impaired his impulse control.
OR MAYBE THAT LAST PART ISN'T TRUE.
Phineas Gage was NOT much changed by this life-altering injury. Though he lost his job at the railway company, the cause of this job loss is unknown. He MAY have had severe alterations to his personality due to this injury, but whether these changes were due to physical damage or emotional trauma--or whether personality changes ACTUALLY occurred at all--are disputed.
----
Can We Even Learn Anything From Gage?
Though I am uncertain if we have exact data on which parts of his brain he was left with when accounting for what was later lost to infection, the trajectory and angle of the injury suggests he initially lost much of his prefrontal cortex. Which of the previous versions of the story are told or over/under-emphasized is dependent on the point the teller is trying to make in the age-old debate of nature vs. nurture.
Some psychologists argue that Gage's personality change demonstrates the Global Workspace Model, where different parts of the brain are responsible for different parts of consciousness, and that by changing or removing parts of the brain, you change consciousness.
Other psychologists will argue that the LACK of change is evidence of the brain's incredible plasticity--its ability to adapt and compensate for missing parts by shifting the functions of those parts to be performed by different regions.
Most reasonably, he probably experienced some cognitive differences while still being effectively the same person and is an example of both points of view. But we don't have concrete enough evidence to say.
Any class in which a teacher or textbook needs evidence to support whatever point they're trying to make about how changes to the brain affects personality, addiction, emotional regulation, decision making, etc., they'll use Gage to make that point, no matter what stance they take. So really, Gage isn't a useful case study beyond what we could actually observe: he lost some of his brain and lived, while also experiencing migraines and seizures for the rest of his life.
With all of that said, if we assume that Gage experienced no changes to cognitive function or personality, I just typed out a story I am very sick of hearing for no reason. So let's assume that at least some of those observations were true.
----
Let's FINALLY Talk About Video Game
Tumblr media Tumblr media
Her are some potentially useful images to reference if you want. Left: general brain regions and their functions. Right: paranasal sinus cavities.
Unlike a nice, straight tamping iron, Mohg's horns curl in unpredictable directions. Some assumptions must be made about length, depth, and diameter to determine what region and volume of his skull is occupied by his horn. The minimum I expect is that the horn occupies the region of his frontal lobe in any scenario. Let's also set a maximum limit: I believe it is reasonable to assume it has not reached the primary motor cortex, where it would disrupt body control and physical movement... unless one wants to suggest he is puppetting himself in his boss fight like a bloodbender. Which, let's be real, IS a really badass concept, someone should write that fanfiction.
Though I argue that Gage is a bad example to use given our lack of reliable data on his personality and lived experiences, we DO know that disrupting the function of the prefrontal cortex can affect judgment, planning, concentration, and any type of higher processing you might call a uniquely 'human' mental ability (I acknowledge the mental abilities of birds and primates but they are beyond the scope of this essay). It may be safe to assume that, in Mohg's case, these mental processes are harmed regardless of any further extrapolation I make. One other brain region of note is the motor speech (Broca) area, located on the left side directly behind the prefrontal cortex and controls muscle movements for speech.
On the topic of pain, migraines, and seizures: He has a horn in his head, it probably hurts. Obstructions (like cysts) can cause buildup of cerebrospinal fluid, which can cause pain and is a common cause of seizures. It is difficult to say how many people have benign brain tumors, but there is speculation that benign tumors in the brain are unexpectedly common. People only typically get brain scans when they've already noticed a problem, but there have been cases of perfectly healthy people having (non-cancerous) brain tumors, so a mass being present in the brain does NOT guarantee seizures will occur. This being said, that horn is significantly larger than a typical benign brain tumor. Migraines and seizures are very reasonable to assume.
I don't know what to say about illness and disease. In theory, if the horn grew at any point after birth, I would say he should have died from any pathogens that were introduced during its corkscrewing into his skull. Phineas Gage was bedridden for months due to infection, was under the care of a doctor, and he wasn't living in a sewer. Do the Lands Between understand the germ theory of disease? It may at least know that poop in the brain is bad, but I listen to Sawbones, so I know that isn't something we can just assume. It's possible he's lost some impossible-to-estimate amount of brain matter to infection. Feel free to speculate about Omen resistance to pathogens, but I don't feel that is the point of this essay. I'll say it's safe to assume his body has healed closed around it, but anything else I won't try to extrapolate.
----
Specificity from Horn Trajectory
Possibility 1:
Tumblr media
If we estimate the continued trajectory from the visible part of the horn, it actually continues medially, towards the center of the body, and curls downward. This might even miss most of the brain and instead disrupt the frontal, ethmoidal, and maxillary sinus cavities of the skull.
It may possibly even pierce the roof of the mouth, if we roughly estimate the rate at which the horn tapers and where it likely ends. I argue that this is the most optimistic scenario in terms of his health, because although the horn almost certainly penetrates the prefrontal cortex, it may not be as deep as other possibilities.
In this horn trajectory case, he probably experiences constant sinus pressure similar to a permanent head cold, obstruction to his sense of smell, and by extension his sense of taste. Even if the horn does not completely block his nasal cavity, it may have damaged his olfactory nerve and thus disabled his sense of smell anyway. Should the horn obstruct his mouth he may experience physical difficulties eating and speaking.
Possibility 2:
Tumblr media
A worse scenario may be to assume this horn instead extends directly backwards. This would likely pass through the motor speech area, and may have caused him to lose the ability to talk, forcing him to relearn how to speak by having another part of the brain learn to do this function (similar to how anyone learns a second language after very early childhood). It may also reach the LEFT temporal lobe, which processes hearing and smell for the RIGHT side of the body, and therefore he could be deaf in his right ear. Again, the olfactory nerve is potentially in the path of the horn, and loss of sense of smell is frequently considered a symptom of brain damage, so regardless of the angle of the horn this is a high possibility.
----
What Time of Horn Growth Could Tell Us
Children are more likely to recover well from brain damage. The older he was when the horn entered his brain, the more likely he would be to experience cognitive impairment.
Should Mohg's horn have developed that way before birth, his brain may have formed around it without issue, or obstructed regions may have simply remained underdeveloped. His skull would also have developed to more 'comfortably' accommodate this horn, rather than having to break and re-heal around a later intrusion. If the horn is shallow enough and its growth occurred during fetal development or very early childhood before the fusing of the bones in the skull, it is possible that left eye blindness and mild discomfort are the only effects. The timing of the horn's growth being before birth or in early infancy is supported by the Regal Omen Bairn, which shows Morgott with seemingly all of his horns, suggesting that omens horns are largely present upon birth and that those horns grow in proportion with them.
However, given the themes associated with the Formless Mother, here is another--vastly more speculative--hypothesis: Mohg's horn was grown deliberately into his skull by the influence of the Formless Mother, perhaps with or without his consent. I find it hard to believe that a force claimed to be the "mother of truth" which "desires a wound" would be unaware of the possible effects of this type of wound.
I posit that the Formless Mother intended to compromise Mohg's consciousness and sense of reason to make him easier to manipulate. If we assume that they were not working together (debatable), the abduction of Miquella and potential interruption and sabotage of his ascension puts an empyrean under the Formless Mother's control, and works counter to the dynasty Mohg desires. Damage to his ability to plan, make rational decisions, and his sense of morality could explain how Mohg seems to want a place for outcast and hated people, likely seeing a kinship with Miquella, but has created something that is the antithesis to the Haligtree.
Furthermore, should we assume that Mohg and Miquella met previously and Miquella had the opportunity to do so, the power Miquella purportedly has to compel adoration in others may have interacted poorly with Mohg's potentially impaired emotional processing, and could have caused an obsessive outcome that the Formless Mother did not predict.
Of course, I don't believe every awful and cruel decision someone makes is the result of brain damage, but this may explain the incongruity between what Mohg seems to want and what he has made. Whether Mohg is "the reigning lord and hierarch of the coming dynasty of Mohgwyn" or "a raving lunatic" may not be an incompatible dichotomy. It may be sequential.
----
Glassbirdfeather you're so wrong, why did you say ___?
I am not a doctor. I am a chemistry student with a biology lean (clinical laboratory science) and am drawing my conclusions from what I've learned in Anatomy, Physiology, and Psychology classes at an introductory level, and I glanced back at my anatomy and psychology textbooks as my sole academic sources. Please don't take this as a well-researched essay, none of the claims I make about mental or physical health are properly cited. This is just fandom theorizing; it's as academically rigorous as fanfiction. Any doctor/member of the medical profession who would like to correct me is invited to do so, I would love to hear more accurate and informed observations.
----
Bibliography
(literally just 2 references, man)
Grison, Sarah and Michael S. Gazzaniga. Psychology in Your Life. Third Edition, W. W. Norton & Company, 2019.
McKinley, Michael P. and Valerie Dean O'Loughlin. Human Anatomy. Fifth Edition, McGraw-Hill, 2017.
392 notes · View notes
nostalgebraist · 3 months
Text
information flow in transformers
In machine learning, the transformer architecture is a very commonly used type of neural network model. Many of the well-known neural nets introduced in the last few years use this architecture, including GPT-2, GPT-3, and GPT-4.
This post is about the way that computation is structured inside of a transformer.
Internally, these models pass information around in a constrained way that feels strange and limited at first glance.
Specifically, inside the "program" implemented by a transformer, each segment of "code" can only access a subset of the program's "state." If the program computes a value, and writes it into the state, that doesn't make value available to any block of code that might run after the write; instead, only some operations can access the value, while others are prohibited from seeing it.
This sounds vaguely like the kind of constraint that human programmers often put on themselves: "separation of concerns," "no global variables," "your function should only take the inputs it needs," that sort of thing.
However, the apparent analogy is misleading. The transformer constraints don't look much like anything that a human programmer would write, at least under normal circumstances. And the rationale behind them is very different from "modularity" or "separation of concerns."
(Domain experts know all about this already -- this is a pedagogical post for everyone else.)
1. setting the stage
For concreteness, let's think about a transformer that is a causal language model.
So, something like GPT-3, or the model that wrote text for @nostalgebraist-autoresponder.
Roughly speaking, this model's input is a sequence of words, like ["Fido", "is", "a", "dog"].
Since the model needs to know the order the words come in, we'll include an integer offset alongside each word, specifying the position of this element in the sequence. So, in full, our example input is
[ ("Fido", 0), ("is", 1), ("a", 2), ("dog", 3), ]
The model itself -- the neural network -- can be viewed as a single long function, which operates on a single element of the sequence. Its task is to output the next element.
Let's call the function f. If f does its job perfectly, then when applied to our example sequence, we will have
f("Fido", 0) = "is" f("is", 1) = "a" f("a", 2) = "dog"
(Note: I've omitted the index from the output type, since it's always obvious what the next index is. Also, in reality the output type is a probability distribution over words, not just a word; the goal is to put high probability on the next word. I'm ignoring this to simplify exposition.)
You may have noticed something: as written, this seems impossible!
Like, how is the function supposed to know that after ("a", 2), the next word is "dog"!? The word "a" could be followed by all sorts of things.
What makes "dog" likely, in this case, is the fact that we're talking about someone named "Fido."
That information isn't contained in ("a", 2). To do the right thing here, you need info from the whole sequence thus far -- from "Fido is a", as opposed to just "a".
How can f get this information, if its input is just a single word and an index?
This is possible because f isn't a pure function. The program has an internal state, which f can access and modify.
But f doesn't just have arbitrary read/write access to the state. Its access is constrained, in a very specific sort of way.
2. transformer-style programming
Let's get more specific about the program state.
The state consists of a series of distinct "memory regions" or "blocks," which have an order assigned to them.
Let's use the notation memory_i for these. The first block is memory_0, the second is memory_1, and so on.
In practice, a small transformer might have around 10 of these blocks, while a very large one might have 100 or more.
Each block contains a separate data-storage "cell" for each offset in the sequence.
For example, memory_0 contains a cell for position 0 ("Fido" in our example text), and a cell for position 1 ("is"), and so on. Meanwhile, memory_1 contains its own, distinct cells for each of these positions. And so does memory_2, etc.
So the overall layout looks like:
memory_0: [cell 0, cell 1, ...] memory_1: [cell 0, cell 1, ...] [...]
Our function f can interact with this program state. But it must do so in a way that conforms to a set of rules.
Here are the rules:
The function can only interact with the blocks by using a specific instruction.
This instruction is an "atomic write+read". It writes data to a block, then reads data from that block for f to use.
When the instruction writes data, it goes in the cell specified in the function offset argument. That is, the "i" in f(..., i).
When the instruction reads data, the data comes from all cells up to and including the offset argument.
The function must call the instruction exactly once for each block.
These calls must happen in order. For example, you can't do the call for memory_1 until you've done the one for memory_0.
Here's some pseudo-code, showing a generic computation of this kind:
f(x, i) { calculate some things using x and i; // next 2 lines are a single instruction write to memory_0 at position i; z0 = read from memory_0 at positions 0...i; calculate some things using x, i, and z0; // next 2 lines are a single instruction write to memory_1 at position i; z1 = read from memory_1 at positions 0...i; calculate some things using x, i, z0, and z1; [etc.] }
The rules impose a tradeoff between the amount of processing required to produce a value, and how early the value can be accessed within the function body.
Consider the moment when data is written to memory_0. This happens before anything is read (even from memory_0 itself).
So the data in memory_0 has been computed only on the basis of individual inputs like ("a," 2). It can't leverage any information about multiple words and how they relate to one another.
But just after the write to memory_0, there's a read from memory_0. This read pulls in data computed by f when it ran on all the earlier words in the sequence.
If we're processing ("a", 2) in our example, then this is the point where our code is first able to access facts like "the word 'Fido' appeared earlier in the text."
However, we still know less than we might prefer.
Recall that memory_0 gets written before anything gets read. The data living there only reflects what f knows before it can see all the other words, while it still only has access to the one word that appeared in its input.
The data we've just read does not contain a holistic, "fully processed" representation of the whole sequence so far ("Fido is a"). Instead, it contains:
a representation of ("Fido", 0) alone, computed in ignorance of the rest of the text
a representation of ("is", 1) alone, computed in ignorance of the rest of the text
a representation of ("a", 2) alone, computed in ignorance of the rest of the text
Now, once we get to memory_1, we will no longer face this problem. Stuff in memory_1 gets computed with the benefit of whatever was in memory_0. The step that computes it can "see all the words at once."
Nonetheless, the whole function is affected by a generalized version of the same quirk.
All else being equal, data stored in later blocks ought to be more useful. Suppose for instance that
memory_4 gets read/written 20% of the way through the function body, and
memory_16 gets read/written 80% of the way through the function body
Here, strictly more computation can be leveraged to produce the data in memory_16. Calculations which are simple enough to fit in the program, but too complex to fit in just 20% of the program, can be stored in memory_16 but not in memory_4.
All else being equal, then, we'd prefer to read from memory_16 rather than memory_4 if possible.
But in fact, we can only read from memory_16 once -- at a point 80% of the way through the code, when the read/write happens for that block.
The general picture looks like:
The early parts of the function can see and leverage what got computed earlier in the sequence -- by the same early parts of the function. This data is relatively "weak," since not much computation went into it. But, by the same token, we have plenty of time to further process it.
The late parts of the function can see and leverage what got computed earlier in the sequence -- by the same late parts of the function. This data is relatively "strong," since lots of computation went into it. But, by the same token, we don't have much time left to further process it.
3. why?
There are multiple ways you can "run" the program specified by f.
Here's one way, which is used when generating text, and which matches popular intuitions about how language models work:
First, we run f("Fido", 0) from start to end. The function returns "is." As a side effect, it populates cell 0 of every memory block.
Next, we run f("is", 1) from start to end. The function returns "a." As a side effect, it populates cell 1 of every memory block.
Etc.
If we're running the code like this, the constraints described earlier feel weird and pointlessly restrictive.
By the time we're running f("is", 1), we've already populated some data into every memory block, all the way up to memory_16 or whatever.
This data is already there, and contains lots of useful insights.
And yet, during the function call f("is", 1), we "forget about" this data -- only to progressively remember it again, block by block. The early parts of this call have only memory_0 to play with, and then memory_1, etc. Only at the end do we allow access to the juicy, extensively processed results that occupy the final blocks.
Why? Why not just let this call read memory_16 immediately, on the first line of code? The data is sitting there, ready to be used!
Why? Because the constraint enables a second way of running this program.
The second way is equivalent to the first, in the sense of producing the same outputs. But instead of processing one word at a time, it processes a whole sequence of words, in parallel.
Here's how it works:
In parallel, run f("Fido", 0) and f("is", 1) and f("a", 2), up until the first write+read instruction. You can do this because the functions are causally independent of one another, up to this point. We now have 3 copies of f, each at the same "line of code": the first write+read instruction.
Perform the write part of the instruction for all the copies, in parallel. This populates cells 0, 1 and 2 of memory_0.
Perform the read part of the instruction for all the copies, in parallel. Each copy of f receives some of the data just written to memory_0, covering offsets up to its own. For instance, f("is", 1) gets data from cells 0 and 1.
In parallel, continue running the 3 copies of f, covering the code between the first write+read instruction and the second.
Perform the second write. This populates cells 0, 1 and 2 of memory_1.
Perform the second read.
Repeat like this until done.
Observe that mode of operation only works if you have a complete input sequence ready before you run anything.
(You can't parallelize over later positions in the sequence if you don't know, yet, what words they contain.)
So, this won't work when the model is generating text, word by word.
But it will work if you have a bunch of texts, and you want to process those texts with the model, for the sake of updating the model so it does a better job of predicting them.
This is called "training," and it's how neural nets get made in the first place. In our programming analogy, it's how the code inside the function body gets written.
The fact that we can train in parallel over the sequence is a huge deal, and probably accounts for most (or even all) of the benefit that transformers have over earlier architectures like RNNs.
Accelerators like GPUs are really good at doing the kinds of calculations that happen inside neural nets, in parallel.
So if you can make your training process more parallel, you can effectively multiply the computing power available to it, for free. (I'm omitting many caveats here -- see this great post for details.)
Transformer training isn't maximally parallel. It's still sequential in one "dimension," namely the layers, which correspond to our write+read steps here. You can't parallelize those.
But it is, at least, parallel along some dimension, namely the sequence dimension.
The older RNN architecture, by contrast, was inherently sequential along both these dimensions. Training an RNN is, effectively, a nested for loop. But training a transformer is just a regular, single for loop.
4. tying it together
The "magical" thing about this setup is that both ways of running the model do the same thing. You are, literally, doing the same exact computation. The function can't tell whether it is being run one way or the other.
This is crucial, because we want the training process -- which uses the parallel mode -- to teach the model how to perform generation, which uses the sequential mode. Since both modes look the same from the model's perspective, this works.
This constraint -- that the code can run in parallel over the sequence, and that this must do the same thing as running it sequentially -- is the reason for everything else we noted above.
Earlier, we asked: why can't we allow later (in the sequence) invocations of f to read earlier data out of blocks like memory_16 immediately, on "the first line of code"?
And the answer is: because that would break parallelism. You'd have to run f("Fido", 0) all the way through before even starting to run f("is", 1).
By structuring the computation in this specific way, we provide the model with the benefits of recurrence -- writing things down at earlier positions, accessing them at later positions, and writing further things down which can be accessed even later -- while breaking the sequential dependencies that would ordinarily prevent a recurrent calculation from being executed in parallel.
In other words, we've found a way to create an iterative function that takes its own outputs as input -- and does so repeatedly, producing longer and longer outputs to be read off by its next invocation -- with the property that this iteration can be run in parallel.
We can run the first 10% of every iteration -- of f() and f(f()) and f(f(f())) and so on -- at the same time, before we know what will happen in the later stages of any iteration.
The call f(f()) uses all the information handed to it by f() -- eventually. But it cannot make any requests for information that would leave itself idling, waiting for f() to fully complete.
Whenever f(f()) needs a value computed by f(), it is always the value that f() -- running alongside f(f()), simultaneously -- has just written down, a mere moment ago.
No dead time, no idling, no waiting-for-the-other-guy-to-finish.
p.s.
The "memory blocks" here correspond to what are called "keys and values" in usual transformer lingo.
If you've heard the term "KV cache," it refers to the contents of the memory blocks during generation, when we're running in "sequential mode."
Usually, during generation, one keeps this state in memory and appends a new cell to each block whenever a new token is generated (and, as a result, the sequence gets longer by 1).
This is called "caching" to contrast it with the worse approach of throwing away the block contents after each generated token, and then re-generating them by running f on the whole sequence so far (not just the latest token). And then having to do that over and over, once per generated token.
302 notes · View notes