Lessons from using web frameworks for personal projects

November 25th, 2023

Software engineers love working on side projects. Side projects are a good way of picking up new skills which would normally fall outside the scope of one's day job. If someone is a DevOps engineer, it's likely they have less experience with front-end technologies. A potentially useful side project would be to write a website. The depth can vary - they could start with simple HTML and CSS, and slowly add JavaScript, start using bundlers etc. They could also decide they want to work on the entire stack, and write the web server component, too! They could also get as low level as they want with the server: use an existing server and customize the routes, write a templating engine, or even implement a part of the RFC from scratch.

I, too, wanted to start a side project. As someone who transitioned from the finance industry, I discovered this newfound passion, manifesting as this unrelenting desire to "do all the things". It's an unattainable goal, of course, but at the same time it's also this never ending source of fuel. You want to learn about everything. You dream about it when you sleep. You think about the different problems you encountered and how to solve them even when you're not in front of a screen or a keyboard. It's a great feeling.

My project of choice was a website (yes, the same one from the first paragraph). Set up a little corner where I could introduce myself, what I'm working on, and also write about some of my thoughts. When I began this project in 2016, it was no longer cool to manipulate the DOM manually, not via jquery or nor via the native browser APIs. The biggest thing was React. Everyone was going crazy for React. The thing with React, however, was that it was this non-intuitive abstraction for writing responsive websites. Not only that, it also introduced this weird syntax with JSX. It went against everything I was taught. All of a sudden we didn't care about the "separation of concerns" and started writing HTML, CSS and JavaScript in a single file.

Here is another fact about the tech industry: it moves fast. Very fast. It's not uncommon to have FOMO when new technologies emerge. And sometimes it's not even FOMO - it's simply not wanting our skills to become obsolete by focusing on technology that is slowly being phased out. I was at an intersection with a few choices, and I wanted to make sure I invest my time learning new technologies which are future proof and will continue having demand in the industry. I cave in and decided to go with React. Here is the outcome (my avatar image is missing as I used wayback machine for this):

Alt text

Although this is a modest website, the effort it took me to build it was not trivial. There were around four React component there, written the old-fashioned way with ES6 classes. There was an awful lot of boiler code just to render this arguably simple page. Nevertheless, the thing worked, and that was my site for the good part of three years. Fast forward to 2019. I wanted to add a blog section to the site. I clone the repo of the site, start the dev server and open the react documentation to see how to go about adding a blog section. Uh oh, it looks like a lot changed between 2016 and 2019 in React - we don't use classes anymore. Hooks are the new thing, they are a "better" abstraction for components compared to ES6 classes. My initial reaction was this feeling of despair. I came here to add a blog section using syntax I thought I know, but instead I now had to rewrite the existing website using the new hooks syntax - before moving on to doing the actual work I came here to do: adding the blog section. I say "had to", but of course there no no real necessity. It was that little voice in my head telling me "Psst, Itay, use hooks because React with classes will soon become obsolete. And you know what happens to engineers with obsolete skills 😈".

You may have read the paragraph above and thought to yourself - why is Itay using React when all he really wants is a static website? That's exactly the thought that went through my mind. Can I keep writing React, but have my code be converted to a static website somehow? Why yes! We are now in the era of static site generators based on React, and Gatsby is the reigning king. I bootstrapped a new Gatsby project. The experience was very similar to create-react-app, with one big difference - we now use GraphQL. What? Yes, GraphQL for a static website. They have a reasonable explanation for why it's used. To be fair to the Gatsby folks, there is a way of avoiding the GraphQL layer for simpler websites, but I didn't know about it at the time. Instead of figuring it all out, I ended up using a Gatsby Starter as a template for my blog. It meant I had less control over the design, but I could at least focus on creating content. Here is what polyglot.codes looked like back in 2020 (that's as far as wayback machine would go):

Alt text

Modifying the design of the Gatsby starter wasn't not easy, as a lot of the configuration and content that would normally fall under code bundled in the main project, is offloaded to the starter, which is a separate npm package on its own. Nevertheless, that approach worked really well for me. All I had to do was write MDX files. Ah, yes, MDX. "Separation of concerns" they said - let's write JSX in Markdown files 😂. Jokes aside, the ability to mix in React components in my blog post, as a part of my Markdown file was pretty cool!

Fast forward a couple of years, I was looking to add more content to the site. New talks, presentations and a new section. I cloned the repo, ran npm run install and npm run build, and see this stream of red text in my terminal. It looks like a lot of the dependencies moved since the last time I build the website, and nothing works anymore. My mistake was not checking in the package-lock.json file. But still, I didn't expect the build to fail so spectacularly. I spent a good day getting everything to build. I knew that I'll need to migrate away from Gatsby one day if I want to avoid this experience again.

That day came in 2023. What could have changed in three years, you ask?

And so the cycle begins yet again 😄. I spent a couple of days building a small website, before realizing doing it properly will take a lot more work than I have time for. I decided to pay for a NextJS website template created by the Tailwind folks. There was a lot to customize there - images, logos, remove pages, add pages, color schemes. So much. I was up for it, though! I spent a two weeks (!) customizing that template. It was beautiful. I ran npm run dev, checked the server booted okay, loaded the website on localhost, checked the pages worked. Everything looked good - we were ready to ship!

Now is a good time to mention that NextJS is sponsored by Vercel. I wanted to avoid using Vercel as the host for this website for a number of reasons, with the most compelling one being avoiding vendor lock-in. It was a matter of principle to be able to choose any host for a website which is meant to be static. It turns out that although it's definitely possible, it's not easy deploying NextJS sites on hosts other than Vercel. Given my site was truly static, I thought it would make sense to deploy it on Cloudflare. I installed the required packages and made the required changes to support running the site on an edge runtime.

Once everything was in place, I clicked on the deploy button and eagerly read the live log feed to see how the deployment progresses. It failed when running npm run build, which I thought was odd, because why would npm run build fail if npm run dev works? I am aware there is a difference, but I expected any big issues to cause the website to fail compiling on both commands. I dug deeper, and the issue was with the function which took the MDX files and converted them to HTML. I didn't touch that function whatsoever, it came with the template I bought. My typical debugging process involves looking at the delta between changes I made and a state I know previously worked. The problem was that I didn't have that reference point - I didn't run npm run build before, I had assumed that any issues would be flagged by npm run dev.

I started undoing all my changes, one by one, and running npm run build after each one. It wasn't the Cloudflare configuration. I googled and saw similar issues reported, potentially stemming from problematic Webpack 5 configurations. I was gobsmacked. It made sense that npm run build would attempt to invoke some bundling process, which could fail, versus npm run dev which could have lighter bundling process. Why would I need to modify my bundler configuration to get past this error? Didn't this configuration work just fine before I made my changes? How are my changes impacting the bundler? Is my assessment that it's a bundler error even correct or am I looking in the wrong place? I spent two more days trying to get the website to build, to no avail. I remember going to bed, thinking to myself: "This shouldn't be so hard. We're building a static website!". When I woke up the next morning, my conclusion was that I am not using the right tool for the right job. If all I need is to generate a static website from some Markdown files, perhaps NextJS (and React!) is an overkill.

My fear of missing out on using the latest web technology cost me. Deep inside I knew that using a full-fat framework may be an overkill for a static website, but I let FOMO get the best of me and cloud my judgement. Frameworks are great when used for the right reasons in the right environment. They make it easier to converge on development patterns, but these patterns also come with a lot of abstractions and, sometimes, the cost of these abstractions is too high for unsuitable projects - projects like my personal site.

Thanks for making it this far! In the next post, I'll be writing about how I built a static website generator - used to build the page you're reading now - from scratch, and in Rust.

Cheers!