When you have more than a handful of pages on your site updating anything shared between them becomes tedious. Things like your site menu, a copyright footer if you have one, perhaps the default header and meta tags.
There were options. You could use a tool like Macromedia Dreamweaver, GoLive CyberStudio, HoTMetaL, or HotDog (one of New Zealand’s .com boom success stories!) to author all your HTML and copy the files you could use their built-in templating features, or copy/paste the changes.
If you weren’t rich enough to use those tools - they did cost quite a bit - you’d either have to roll your own using local scripts, or rely on server-side features.
The most obvious is PHP. This is one of the reasons PHP was created, PHP originally stood for Personal Home Page and was a CGI script. The original PHP syntax was very different to what ended up being PHP3, in fact it was very similar to what I’m about to use.
I’ve updated my website with all the pages, common elements brought to you by Server-side Includes!
Server-side Includes
These date back to the NCSA httpd from 1993, one of the first web servers, and are still supported by modern servers like nginx. At their most basic server-side includes, or SSI, are web server feature that looks for special HTML comments and replaces them with the contents of another file.
<!--#include virtual="./header.html" -->
This tells the web server to resolve the path ./header.html and substitute it in. virtual means use the HTTP path resolution rather than filesystem paths. Most web servers prohibit using file with an absolute path or with any ../ in the path. This is because the web server often has permission to read many other user’s home directories, so it’s possible to get around file permissions with this.
virtual also allows you to reference CGI scripts. I could have implemented my counter to return text and used <!--#include virtual="~thea/cgi-bin/counter.cgi" -->.
Files included this way may also be processed for SSI themselves, resulting in several levels of include if you go that far. Most servers have a limit on how deeply nested includes can be.
Apache also allows you to configure CGI scripts to have their result processed by the SSI module, though this is not the default setting. Apache also has an additional option called XBitHack, which lets you set the execute bit on your HTML rather than renaming to .shtml. You can also configure it to process all files for include directives, but this has performance implications. Or at least it did in 1999.
Variables
Typically the query string and some request and server details are exposed as variables.
Apache’s implementation SSI has rudimentary support for setting variables, <!--#set var="title" value="About" -->. Interpolation is also available to build variables out of other variables. You can then <!--#echo var="title" --> to get the value.
SSI does not have any form of namespacing, but whether variables set in one file are visible in another depends on the server. In Apache they are, so I used variables to pass down the title. Normally included pages have access to variables set in the parent.
Conditionals
Most SSI implementations have basic conditionals - <!--#if -->, <!--#else -->. I haven’t had the need to use them, but they could be used to include different files based on browser user-agent. This was quite handy when you needed very different styles for IE and Netscape.
As many servers supplied the query string to the page you could make a slightly dynamic page using SSI conditionals and query strings. Some nefarious people used this to implement loops
Error handling
[an error occurred while processing this directive]
That’s pretty much all you get! Unless you’re the server admin and can check the logs you’re kind of stuck.
And that’s it
Server-side Includes are a very basic tool, but they can be very powerful. However we moved on to more powerful things, like the aforementioned PHP, which I may just cover soon.
The next instalment in building a website like it’s 1999 - adding some interactivity! After all if people can’t submit things on your website how else will you hear from them?
I’m going to add two critical parts of a 90s website - a hit counter and a guestbook.
The guestbook page now works! It has a completely different style from the homepage, as was the fashion at the time. You have to show off all your design skills.
Running code on the server
In 1999 you couldn’t just run code on the server. Most hosts only allowed you FTP access to upload files, there was no way to start a process. The two most common ways to run server-side code were PHP and CGI. PHP3 had been around for a couple of years and had some features that made it really suited for shared web hosting, but I’m not going to use PHP (yet).
Shared hosts would move from allowing CGI to only allowing PHP for some of the reasons I’ll explore. PHP’s big selling point was how well it integrated with HTML, and being less annoying than CGI.
CGI
Common Gateway Interface, CGI, is a specification for web servers to execute programs and pass in details about the request. The program will then output an HTTP response with a few special headers, and the response is forwarded on to the client.
CGI programs could be anything the host system could execute, they can be compiled programs but since most of the time shared hosts did not allow you to compile code various interpreted languages were used. The most common was Perl, even though there were widely available alternatives like Python. Perl was commonly used by sysadmins and therefore usually available, had many modules installed, and fast. Having modules installed was important, your disk space was limited so you’d want as much stuff installed at the system level as possible.
The big gotchas
The biggest gotcha is CGI scripts run as the webserver’s user. Most providers did not give you access to a database, so all data had to be stored in files in your user directory. This has two big implications - first that you have to give the webserver permission to write files, which is accomplished by setting the file or directory’s group to the webserver’s group (www in my case), and secondly that any other user’s scripts can write to those files as well.
There are solutions to this. One is setting the setuid bit on the file which makes it execute using the file’s owner uid instead of the caller’s, but this requires remembering to do it every time. Another is using CGI wrappers that run as root and change to the appropriate user before executing the script, but those have potential security vulnerabilities.
Thea’s Script Archive
Most people didn’t write their own scripts. We got them from Matt’s Script Archive, which is amazingly still online and still serving the old scripts!
I didn’t use those scripts. Matt’s scripts are from a different time, where the answer to “You shouldn’t put the destination email address in a hidden form field, someone might change it and be able to send email to anyone!” was “Who would want to do that?”. They often deliberately do things we’d consider a security vulnerability today, like trusting user input (formmail.pl was well known as a spammer’s dream for this) and not sanitising HTML. They also use programs which aren’t available anymore for the same reasons.
So introducing Thea’s Script Archive. They have fewer vulnerabilities! (fewer security vulnerabilities not guaranteed).
The Scripts
While these aren’t genuine 1999 scripts, they have the same base principles. I’ll break down the code behind these scripts in a second post.
Counter
This is a simple hit counter. Every time it’s loaded the count is incremented, there’s no attempt to ignore bots, indexers, or repeated loads by the same person. These features would start to be added to counter, and for people who couldn’t use CGI scripts third party services existed to supply these. “Real” analytics were normally performed on the server access log files using programs like (Sawmill)[http://www.sawmill.net], which was still in business until 2021.
The image is created on the fly, there’s no caching here!
Guestbook
This is even simpler than the counter. Like many similar scripts from its time it doesn’t even use a database - if you only have FTP access to the server you can’t update a database to remove spam! Instead it simply appends to the HTML file using a specific comment to locate where to add the entry.
Some scripts allowed you to use a template hidden in the comment, but this one is very simple. HTML is escaped though.
Yesterday’s post was getting a bit long, so for those who weren’t doing web design in the late 90s/early 2000s here’s some background.
The constraints of 1999
The late 90s were a time of rapid change in the world wide web, the first big browser war was brewing. Netscape Navigator browser got bloated with the Communicator product they were pushing and Microsoft’s bundling of Internet Explorer with Windows rapidly pushed Netscape out of the market. Both implemented parts of HTML4 and CSS 1 and 2 in differently buggy ways, and they both had different ways of using JavaScript to animate elements on the page. Back then we called it Dynamic HTML!
The main constraints I’m following are:
CSS
CSS! Yes that’s right, we had CSS in 1999! CSS 1 support was reasonable, text and element alignment, text and background colour, and font face were reliable. Padding and margin worked on some elements. CSS2 support which included floats and positioning were not really ready.
Most existing sites still used <font> tags, but CSS was starting to be accepted. CSS let you alter text formatting on links (removing the underline was controversial to say the least) and IE4 introduced the :hover selector on a tags, allowing for mouse-over colour changes. Yes, bold-on-hover was a real thing we did!
Page layout
Layout choices are very limited. There’s no way to break an element out of the document flow (no floats, no CSS positioning), and there are only two ways to have multiple regions on a page.
First we have frames. Frames let you divide the window in to multiple separate named documents, so you could have a left frame 200px wide called ‘menu’ containing the site menu, and the rest of the document as a frame called ‘main’. <a target="main"> in the menu would cause the link to open in the main frame. This still exists as the <iframe> element, but frames are long deprecated. I won’t use frames in the main series, perhaps as an aside.
The other was the (mis)use of the <table> element. You’ll see this soon.
Fonts
Only fonts pre-installed on the user’s system could be used, no web fonts! In the early days this meant taking a guess and hoping. In the mid 90s Microsoft released their Core Fonts for the Web project, which was a bundle of fonts licensed for non-commercial use available for download. There was some traction getting them bundled on non-Windows systems, but it took a long time. Generally multiple fallbacks were provided.
Colour choices
Even though most people in 1999 had video cards that supported at least 24-bit “true colour” at 800x600 people using 16-bit “high colour” were still around. Due to how different systems mapped the colourspace it was common for colours that were distinct on one system to be the same on another.
Images
Even with a storage limit of 10MB this wasn’t the major restriction. Most of the world used dialup modems, with speeds of 56kb/s reasonably common. Even this extremely lightweight website would take around 10 seconds to download (though without custom fonts it’d probably be sub second), and the front page of English language Wikipedia would take close to two minutes.
This means decorative images had to be kept small to avoid stuttering loads - under 10kb each. SVGs were still a few years away, so we have GIF, JPEG and PNG.
Additionally browsers often only allowed a few requests in parallel (2-6 depending on your browser) so using multiple images would result in a site that slowly drew as each image downloaded in turn. Each request needed a whole new TCP connection, in New Zealand fetching from the US that would result in a minimum time of 250ms per image.
Animations
A memory surfaced during this project. In 1999 animations stopped when you scrolled the page. There often wasn’t enough video memory for off-screen rendering, so the whole content area was re-painted on every scroll action. If you dragged the scroll bar the document didn’t drag in real time, but if you pushed the down-arrow repeatedly animations would effectively stop as the timer was disabled during the re-render.
Character set
Unicode definitely exists, but it’s not in use. The ISO 8859-1 character set is generally available, though the Windows codepage 1252 is often actually used. Non-ASCII characters are rare outside of personal sites due to cross-platform and cross-language rendering problems. I’ll be sticking to ASCII characters.
I’m not going to keep up the narrative style from the last post, so from now on it’s back to first person.
In this post I’ll go through the design updates, and I’ll post a follow-up detailing the constraints that went in to this.
Lets get designing!
First, since basic CSS worked in 1999 I’ll add some CSS inside the <head> tag to make the background trendy black and text awesome white, and set the font to something sans-serif for the body and something more impactful for the headings.
I’ve also obtained some page dividing gifs from the internet archive! There aren’t many ways to divide a page up, so images like this are used to visual separate parts of a page.
Using center tags because I don’t recall using <div> tags that early. I think it might have been one of those Netscape vs IE things? I know Netscape had the <layer> tag.
Now to add some links to the future pages of the site. The pages aren’t there yet, but that’s OK because it’s under construction right? They’ll just 404 for now
Now let’s add a fancy animated star background by adding <body background=background.gif>. Yes, CSS for this worked, but what’s more 1999 than mixing styling systems?
Now we can’t really read the links using the default blue and purple. A lot of people wouldn’t be bothered but we can fix it with some more CSS! And add a hover effect for people using IE4!
Next up it’s time for some interactivity with a guestbook!
Author’s notes
Corrections to part 1
In my first post I neglected an important detail - the background colour! In the 90s - at least when using Netscape Navigator and earlier versions of Internet Explorer - the default background colour wasn’t white, it was a mid grey colour. #C0C0C0 to be precise. Why? I’m not entirely sure. I’ve fixed this.
Web browsers are amazingly backwards compatible, almost everything from 1999 still works (except <blink>). There is a problem though - window size. It’s hard to find statistics from the era, but a substantial number of visitors would have been using 800x600 monitors, with 1024x768 also being common. By comparison my 15“ laptop has an effective screen resolution of 1680x1050, and my desktop monitors are even larger.
So I’ve decided I’m going to add some CSS to cap the document width at 1000 pixels and centre it in the browser. Though to be honest this was a common thing to do in 2008 when larger screen sizes appeared and broke the layouts.
So to follow along at home with my website like it’s 1999 series you’ll need a webserver like it’s not 1999. A genuine 1999 webserver would last about 15 minutes on the modern internet.
To set up your own server you’re going to need some degree of experience with UNIX-like systems, DNS, and domain management. I’m not sure if there are many tutorials on this out there, I learnt them a long time ago.
I’m using a virtual machine on my home server, because I’m the sort of person who has a home server. But here’s what I did:
Create a virtual machine to run the server
You could use whole physical machine if you have one handy. I created a VM with 12GB of disk space, 512MB of RAM and 1 vCPU.
An ISP would be able to afford a system with more disk space than this in 1999, but a single core of my 3.5Ghz Intel core i5 4690 would far outclass anything an ISP ran in 1999, even if it is 10 years old at this point.
Install FreeBSD 14.1
Why FreeBSD? I remember it being quite common at the time, and the first UNIX-like server I had a shell account on was FreeBSD. Linux was already well established in 1999, but a lot of the people still regarded it as an upstart that still needed to prove itself. FreeBSD is actually younger than Linux by a couple of years, but its BSD heritage gave it more weight in some opinions.
Also for this series I feel being slightly foreign to most people is an advantage.
I’m not going to include a guide for the installer, the FreeBSD website has documentation.
You can also run Linux, Alpine has everything we need available.
Create a non-root user
useradd my_user. Just remember to add it to the wheel group so you can use su when you SSH in.
Install critical administration utils with pkg
Yes I could use ports, but pkg is good enough.
pkg install vim
Install Apache httpd
pkg install apache24
I’m using Apache despite there being more modern options that are probably better for almost any other use, like nginx and lighttpd. However Apache still supports cutting-edge 90s features that most other servers have removed or never supported, like server-side include processing, CGI script support, and .htaccess support for per-directory configuration overrides. These will come in handy.
Configure Apache
In /usr/local/etc/apache24 edit httpd.conf and uncomment the LoadModule lines for include_module and userdir_module. Also uncomment the line Include etc/apache24/extra/httpd-userdir.conf.
Done!
Yep, that’s pretty much it. The root site is in /usr/local/www/apache24/data.
What’s missing?
SSL
For that true 1999 experience SSL is terminated outside the web server. An ISP in 1999 wouldn’t offer SSL as standard, it used a lot of CPU and required a dedicated IP address per domain name (mostly).
If you’re following along you should use Let’s Encrypt and certbot to configure SSL for you.
Firewall setup
You should think about this, especially if you’re intending to allow other users access to your server. You probably don’t want random servers running.
Things I may add for the future:
Email hosting, SMTP and local delivery at least, probably not IMAP or POP3 unless I move it off my home server.
A database, probably PostgreSQL. I know MySQL was more favoured at the time, but I prefer Postgres.
Perl and python modules as required to run the demos I have planned.
To change from my regular ramblings about healthcare and reading one to many rose-tinted reckons about how things used to be, let’s make a website like it’s 1999!
What this is and isn’t
My aim is to show how you got your site online in 1999, but not to be historically accurate, so
I will be using modern HTML and CSS. Quirks mode belongs in the past
The server software will be modern, because if it wasn’t the server would already be compromised.
I will be using secure methods rather than accurate ones, but I’ll note where this is different.
But I will show how we made websites dynamic and the compromises made when you had limited space!
I’ll make another post on the server setup if you want to follow along. Ask nicely and I might just give you access to TheaNET.
Setting the stage
It’s 1999! You’ve just signed up for your first internet connection with your local ISP, TheaNET! Hey that’s your name as well! They’re a very modern ISP, supporting your new V92 56k modem, and you get 30 hours of connection per month for a very reasonable $29.95.
Looking through the signup pack they sent you, there’s a CD with Netscape Communicator and a bunch of trial apps, some pamphlets, and a letter with connection details on it. One part catches your eye:
All accounts come with complementary website hosting with 10MB of disk space and 100MB of traffic per month.
Not only can you browse all the sites on the internet, you can make your own. This is truely the future.
A few weeks later
How hard can it be? You’ve gone to the local bookstore and purchased a copy of “HTML Made Easy”. The first chapter tells you to open notepad, enter this text, and save it to index.html on your desktop
<html>
<head><title>Thea's Website</title></head>
<body>
<h1>Thea's Website</h1>
<p>Welcome to Thea's website!</p>
<p>This site is still under construction, please check back soon!</p>
</body>
</html>
Now to upload it. The instructions from TheaNET say
To upload to your website, use a program like WinSCP (included on the CD-ROM) or a similar sftp tool. Copy the files in to the public_html directory (if you can’t see it, you can create it). Use your dialup login and password to access your web hosting.
After some fiddling around with the copy program you get the public_html directory created and index.html copied over. Entering https://www.thea.net.nz/~thea/ in to your web browser you see your website come to life! On the internet!
It doesn’t look great, but it’s a start, it says it’s under construction so everyone knows you aren’t finished, and you’re about to read the chapters on images and text formatting.
… to be continued!
Editors Notes
scp instead of ftp
Because I want to make this something safe for others to follow I use scp for file copying. While I could set up an FTP server not only are maintained and secure FTP servers pretty rare, the protocol really doesn’t work well with modern networking. The server I’m using does not have a public IP address.
Generally ISPs did not include shell access with hosting and only allowed ftp access, which is possible with scp as well.
What’s with ~thea?
This is a user directory! In 1999 using a per-user subdomain was not very common, and on UNIX-like systems ~ is used to represent the home directories where users store their files. ~ is your home, ~thea is Thea’s home. The Apache web server continued this convention to represent each user, but served out of public_html in that user’s home directory.
This disappeared as websites got more dynamic due to cookies. Cookies are stored per-domain, so if you had http://www.thea.net.nz/admin/ which set an admin cookie, http://www.thea.net.nz/~nefarious_user/cookie_stealer.html would be able to steal it and gain admin access.