Hello Festive Writers of E2,
I wanted to catch you up over what has been going on over the last month in coding changes on the site, but I have a tale to tell. One of intrigue and dastardly intent. A billionaire versus a scrappy upstart. You see:
Elon Musk Tried to kill E2
Not really, let me explain
The theme of these changes was "Jay didn't understand the documentation, but now he does!"
I recently fixed an interesting bug involving the once-Twitter now X symbol (𝕏). We had a systematic problem where the database did somewhat recently fully support utf-8 characters, but only 3-byte characters. This is because I didn't understand what needed the fourth byte, so I kind of just ignored it. We also didn't have any real bugs associated with it since we basically are an English-only site, and it never came up, until a few folks started talking about Twitter. The new symbol is was called Mathematical Double-Struck Capital X, and came into the unicode standard in 2001, but as you can imagine has really only come into real usage recently. As such, I've stomped out what I am hoping are the last of the utf-8 encoding bugs everywhere and have converted every database table over to the 4-byte format globally, eliminating even number-only tables with latin1 default encodings. We're using utf8mb4_0900_ai_ci which means:
- "utf8mb4" - Every utf8 character currently in existence
- "0900" - Newer engine for it
- ai - Accent independent
- ci - Case independent
This should impact search and mean that you can search for things like Twitter via just the letter X, and have you get to the right thing (sorta). This change is global and I've fixed some of the other utf-8 shims in chat that were breaking it. We're going to have one more round of fixes when legacy javascript retires and we don't need to store html-safe characters in the database, but that's a problem for a few months from now.
We're broadly nearing the end of the necessary infrastructure fixes for the long-term stability of the site. I've stabilized three major scaling issues with the site:
- Scaling out now works properly - We manage load and if load goes back down, we shrink to a comfortable and performant level. Again, I didn't know what that parameter did in Fargate, so that was an embarrassing 10 minute fix.
- We no longer spiral out of control when changes are being pushed or during scaling events. This was because the SpareServers values in Apache were way off, and I didn't tune them properly to manage spares. During load, we could spike in and kill the machine since it too violently went after resources. Now we just size it to the appropriate instance and smoothly manage the threads for any memory leaks
- We have the right memory/CPU ratio: After playing with the numbers for a few days, I have the right blend for the current application for how much each thread takes up in memory on the current architecture. This again is a scaling and load balancing thing, but is basically a cost-thing. Memory units are paid for by the hour, and I don't want to have around more than we need.
Getting these infrastructure things correct is important to the site's long-term health. Right now it costs a few dollars a day for E2 to run. We've entered into a long-term commit with Amazon for the database portion of the services, and will also do so for the container portions now that the scaling is stable. This will cut those costs in half and go a long way to making sure that our ad revenue keeps us in the black while we continue to build.
Now that infrastructure is buttoned up, we're going to continue interface modernization. We're going to continue to be pulling code into the React front-end as well as modernizing the CSS building infrastructure. Everything should eventually become a part of the WebPack asset pipeline in node, instead of living outside of the codebase as a part of the older CSS structures.
So what does this all mean? We're finally at interface work! It'll be doing some more code cleanup to be able to make those structures a bit better, and then we'll be off towards tearing out the nodelet system and moving things away from the old ajax structures and towards React. This is long overdue, but it gives me a surgical point to carve the inside out of E2 without changing it too terribly much while I hack on it. Nodelets will take a couple of months now that infrastructure is good, and then it will be the beginning of the CSS refactor. The last of CSS support will probably die during that time, and it'll be on to new interface work.
If stuff is busted, Let me know!, and I usually fix it within a day or two.
Have a safe and Happy New Year all,
-Jay