When you have more than a handful of pages on your site updating anything shared between them becomes tedious. Things like your site menu, a copyright footer if you have one, perhaps the default header and meta tags.
There were options. You could use a tool like Macromedia Dreamweaver, GoLive CyberStudio, HoTMetaL, or HotDog (one of New Zealand’s .com boom success stories!) to author all your HTML and copy the files you could use their built-in templating features, or copy/paste the changes.
If you weren’t rich enough to use those tools - they did cost quite a bit - you’d either have to roll your own using local scripts, or rely on server-side features.
The most obvious is PHP. This is one of the reasons PHP was created, PHP originally stood for Personal Home Page and was a CGI script. The original PHP syntax was very different to what ended up being PHP3, in fact it was very similar to what I’m about to use.
I’ve updated my website with all the pages, common elements brought to you by Server-side Includes!
Server-side Includes
These date back to the NCSA httpd from 1993, one of the first web servers, and are still supported by modern servers like nginx. At their most basic server-side includes, or SSI, are web server feature that looks for special HTML comments and replaces them with the contents of another file.
<!--#include virtual="./header.html" -->
This tells the web server to resolve the path ./header.html and substitute it in. virtual means use the HTTP path resolution rather than filesystem paths. Most web servers prohibit using file with an absolute path or with any ../ in the path. This is because the web server often has permission to read many other user’s home directories, so it’s possible to get around file permissions with this.
virtual also allows you to reference CGI scripts. I could have implemented my counter to return text and used <!--#include virtual="~thea/cgi-bin/counter.cgi" -->.
Files included this way may also be processed for SSI themselves, resulting in several levels of include if you go that far. Most servers have a limit on how deeply nested includes can be.
Apache also allows you to configure CGI scripts to have their result processed by the SSI module, though this is not the default setting. Apache also has an additional option called XBitHack, which lets you set the execute bit on your HTML rather than renaming to .shtml. You can also configure it to process all files for include directives, but this has performance implications. Or at least it did in 1999.
Variables
Typically the query string and some request and server details are exposed as variables.
Apache’s implementation SSI has rudimentary support for setting variables, <!--#set var="title" value="About" -->. Interpolation is also available to build variables out of other variables. You can then <!--#echo var="title" --> to get the value.
SSI does not have any form of namespacing, but whether variables set in one file are visible in another depends on the server. In Apache they are, so I used variables to pass down the title. Normally included pages have access to variables set in the parent.
Conditionals
Most SSI implementations have basic conditionals - <!--#if -->, <!--#else -->. I haven’t had the need to use them, but they could be used to include different files based on browser user-agent. This was quite handy when you needed very different styles for IE and Netscape.
As many servers supplied the query string to the page you could make a slightly dynamic page using SSI conditionals and query strings. Some nefarious people used this to implement loops
Error handling
[an error occurred while processing this directive]
That’s pretty much all you get! Unless you’re the server admin and can check the logs you’re kind of stuck.
And that’s it
Server-side Includes are a very basic tool, but they can be very powerful. However we moved on to more powerful things, like the aforementioned PHP, which I may just cover soon.
As promised, I’m going to go through how the CGI scripts work. This is not a Perl how-to, or even an example of good perl, but more a description of the process of writing any CGI script.
#!/usr/local/bin/perl
use CGI;
use GD;
use GDBM_File;
The first line, called the shebang, specifies the interpreter to use for the file - this is a UNIX-like feature where any file can be ‘executable’ by specifying the interpreter to launch. In my case as I’m using FreeBSD the perl interpreter is in /usr/local/bin/perl. If you’re familiar with *NIX systems you might wonder why this isn’t /usr/bin/env perl, and it’s not because FreeBSD doesn’t have env. Scripts run in the context of the web server, and the web server does not have /usr/local/bin in its PATH environment variable.
Then we import a bunch of modules we’re going to use. Modules were almost always installed by the sysadmin and you dealt with what was available. However there were a bunch commonly installed, like CGI and GDBM. GD was less common, but was often available on dedicated CGI hosts.
Set up the font and the location of the data file, which must be web server writable or in a directory that is web server writable. It has to be an absolute path otherwise it’d be relative to the web server’s home, which on most systems is created read only.
Then initialise CGI. Perl allows you to omit parenthesis in some cases so CGI->new is actually a method call to create a CGI object. Also grab the page name out of the query string using the new CGI object. Perl actually has both || and or as boolean or operators, they differ only in precedence. This can be handy sometimes, and make things more confusing for the next person.
$db = tie %hash, 'GDBM_File', $datafile, GDBM_WRCREAT, 0640
or die "$GDBM_File::gdbm_errno";
$count = $hash{$page} or 0;
$count++;
$hash{$page} = $count;
$db->sync;
untie %hash;
$db->close;
This is where the counting happens. tie is a perl built-in that creates a hash (aka hashmap or dictionary in other languages) backed by some sort of store. In this case GDBM_File, which uses the GNU implementation of the DBM (DataBase Manager) key-value store. This is a fast store that uses file locks to synchronise access, so we pass GDBM_WRCREAT as the access mode - for Write, Read, Create if not found, in mode 0640, or user read/write, group read, everyone no access.
You may notice that hash is first declared as %hash, then later used as $hash. Unlike other languages like Javascript and PHP Perl does not just use $ to mean variable, there are five possible “sigils” to use the perl term. The ones I’ve used are $ for scalar variables, @ for array variables, and % for hash variables. The sigil indicates how to access the variable, not what type of variable it is. As seen here if you’re talking about the whole hash it’s %hash, but if you’re trying to get a single item from the hash it’s $hash. Sigils are one of the things that make perl very confusing.
Then we update the count, sync the DB, then untie and close it so another request can be processed.
Next draw the count graphic using GD. GD is a graphics drawing library with bindings for multiple languages. Here we create a 100x28 pixel image, set up a black and white colour then draw the counter. sprintf is similar to the C function and here it’s used to output the text with six digits, padded with leading zeros.
The image background is the first colour allocated, so there’s no need to draw the black background.
Finally print the CGI headers and then the GIF itself. The header call syntax is a way to provide a hash argument, it’s a bit weird but that’s Perl for you.
Discussion
I made a few choices here, firstly using DBM for storing the counts. It was common to use a comma or tab delimited file instead, or use a single file per counter. GDBM takes care of concurrency for me, so I used that.
Next in choosing GIF. GD supports many output formats, including modern ones. but I used GIF for old time’s sake. Technically back in 1999 it’d be more likely to use PNG as the patents on GIF caused support to be removed from a lot of libraries, including GD where GIF support wasn’t restored until 2004. However a lot of people used old versions of GD with GIF support remaining.
It could do more, like assign cookies and only count each person once per day, as I’m not in the EU I don’t need to worry about permission, or track data server side to try to reduce duplicate counts, but then the number would be lower! GDBM is a key-value store so it’s easy enough to add, iterate, and look up entries, but it only supports storing scalar values so we can’t store an array or hash without serialising it manually.
But there’s a web counter in under 30 lines of code!
guestbook.cgi
#!/usr/local/bin/perl
use strict;
use CGI;
use POSIX;
use Fcntl qw(:flock);
use HTML::Escape qw/escape_html/;
shebang and module imports. Here we use strict, which will throw an error if we use certain Perl constructs that are common pitfalls, and yes this is where 'use strict'; in Javascript comes from! I didn’t use this in the counter script.
HTML::Escape is a modern module but it isn’t 1999 now so I’m going to escape HTML.
my $url = '/~thea/guestbook.html';
my $file = '/home/thea/public_html/guestbook.html';
my $cgi = CGI->new;
Set up some variables, like file paths and the URL of the guestbook file. And here we see the first strict change - the my keyword. my puts the variable in lexical scope, without my the variable will be in global scope no matter where it’s declared. This is exactly like the behaviour of Javascript in non strict mode with var.
if ($cgi->request_method eq "POST") {
my $name = escape_html $cgi->param("name");
my $message = escape_html $cgi->param("message");
my $website = $cgi->param('website') or '';
If the HTTP method is POST then we grab the parameters. param will fetch from either query string or post body, which is normally considered bad practice today. Additionally we run escape_html on the name and message fields, again omitting the parenthesis from the function call.
Next we do two things, replace new lines with HTML line breaks, and remove unknown characters from the website field. I could use escape_html here, but I though I’d show a more 90s method. This demonstrates Perl’s =~ operator, which sets the var to the result of the regexp substitution on the right.
Grab the time, for the timestamp. This is the POSIX strftime function, so the formatting operators depend on your operating system.
open(FH, '+<', $file) or die 'Cannot open guestbook file';
flock(FH, LOCK_EX) or die 'Cannot lock file';
Open the guestbook file and take an exclusive lock on it. A wiser solution would to loop this in case of concurrency issues, but is that really going to happen?
File handles in Perl are a special type. They can be assigned to regular with my $fh, but that’s not required. Just remember that FH is in global scope.
Read the file in to @content. This is the first time we see a Perl array, and this construct reads the whole file line-by-line in to @content. Then we seek to the start of the file and truncate it.
for (@content) {
if (/<!--NEXT_ENTRY-->/) {
Next loop over the file until we find the next entry marker. You may wonder where the loop variable is, and notice there’s no variable specified in the if. Perl has an implicit variable - $_ - which is automatically used in many functions if you don’t specify a variable. So in this case $_ is the loop variable and the if automatically test against this. $_ is addressable as a regular variable as well.
print FH "<p>$message</p>\n";
if ($website) {
print FH "<p><a href=\"$website\">$name</a> at $time</p>\n";
} else {
print FH "<p>$name at $time</p>\n";
}
print FH "<hr>\n\n"
This just prints the new entry in to the guestbook
}
print FH;
}
close FH;
And finally print the current line with the implied $_ argument, the close the file handle. print to a filehandle doesn’t automatically use $_ if you use a variable-bound filehandle.
Finally redirect back to the guestbook page! Under 50 lines of code!
Discussion
This method of directly modifying the file was quite common. Given most of the time you couldn’t access the server other than via FTP this allowed you to moderate the guestbook by downloading, editing, and re-uploading.
It was also pretty normal to ask for and publish an email address, but then spam stopped this practice.
So what next?
Well this was my foray in to CGI scripts! Next up some other 90s classics are up, like a webring!
I find it interesting how Javascript and Perl both had global by default variables and use strict to prevent those footguns, but somehow Javascript ended up with multiple null types (Perl doesn’t have null, only undef for undefined) and automatic semicolon insertion. Perl has many more features to make incomprehensible code though.
The next instalment in building a website like it’s 1999 - adding some interactivity! After all if people can’t submit things on your website how else will you hear from them?
I’m going to add two critical parts of a 90s website - a hit counter and a guestbook.
The guestbook page now works! It has a completely different style from the homepage, as was the fashion at the time. You have to show off all your design skills.
Running code on the server
In 1999 you couldn’t just run code on the server. Most hosts only allowed you FTP access to upload files, there was no way to start a process. The two most common ways to run server-side code were PHP and CGI. PHP3 had been around for a couple of years and had some features that made it really suited for shared web hosting, but I’m not going to use PHP (yet).
Shared hosts would move from allowing CGI to only allowing PHP for some of the reasons I’ll explore. PHP’s big selling point was how well it integrated with HTML, and being less annoying than CGI.
CGI
Common Gateway Interface, CGI, is a specification for web servers to execute programs and pass in details about the request. The program will then output an HTTP response with a few special headers, and the response is forwarded on to the client.
CGI programs could be anything the host system could execute, they can be compiled programs but since most of the time shared hosts did not allow you to compile code various interpreted languages were used. The most common was Perl, even though there were widely available alternatives like Python. Perl was commonly used by sysadmins and therefore usually available, had many modules installed, and fast. Having modules installed was important, your disk space was limited so you’d want as much stuff installed at the system level as possible.
The big gotchas
The biggest gotcha is CGI scripts run as the webserver’s user. Most providers did not give you access to a database, so all data had to be stored in files in your user directory. This has two big implications - first that you have to give the webserver permission to write files, which is accomplished by setting the file or directory’s group to the webserver’s group (www in my case), and secondly that any other user’s scripts can write to those files as well.
There are solutions to this. One is setting the setuid bit on the file which makes it execute using the file’s owner uid instead of the caller’s, but this requires remembering to do it every time. Another is using CGI wrappers that run as root and change to the appropriate user before executing the script, but those have potential security vulnerabilities.
Thea’s Script Archive
Most people didn’t write their own scripts. We got them from Matt’s Script Archive, which is amazingly still online and still serving the old scripts!
I didn’t use those scripts. Matt’s scripts are from a different time, where the answer to “You shouldn’t put the destination email address in a hidden form field, someone might change it and be able to send email to anyone!” was “Who would want to do that?”. They often deliberately do things we’d consider a security vulnerability today, like trusting user input (formmail.pl was well known as a spammer’s dream for this) and not sanitising HTML. They also use programs which aren’t available anymore for the same reasons.
So introducing Thea’s Script Archive. They have fewer vulnerabilities! (fewer security vulnerabilities not guaranteed).
The Scripts
While these aren’t genuine 1999 scripts, they have the same base principles. I’ll break down the code behind these scripts in a second post.
Counter
This is a simple hit counter. Every time it’s loaded the count is incremented, there’s no attempt to ignore bots, indexers, or repeated loads by the same person. These features would start to be added to counter, and for people who couldn’t use CGI scripts third party services existed to supply these. “Real” analytics were normally performed on the server access log files using programs like (Sawmill)[http://www.sawmill.net], which was still in business until 2021.
The image is created on the fly, there’s no caching here!
Guestbook
This is even simpler than the counter. Like many similar scripts from its time it doesn’t even use a database - if you only have FTP access to the server you can’t update a database to remove spam! Instead it simply appends to the HTML file using a specific comment to locate where to add the entry.
Some scripts allowed you to use a template hidden in the comment, but this one is very simple. HTML is escaped though.
Yesterday’s post was getting a bit long, so for those who weren’t doing web design in the late 90s/early 2000s here’s some background.
The constraints of 1999
The late 90s were a time of rapid change in the world wide web, the first big browser war was brewing. Netscape Navigator browser got bloated with the Communicator product they were pushing and Microsoft’s bundling of Internet Explorer with Windows rapidly pushed Netscape out of the market. Both implemented parts of HTML4 and CSS 1 and 2 in differently buggy ways, and they both had different ways of using JavaScript to animate elements on the page. Back then we called it Dynamic HTML!
The main constraints I’m following are:
CSS
CSS! Yes that’s right, we had CSS in 1999! CSS 1 support was reasonable, text and element alignment, text and background colour, and font face were reliable. Padding and margin worked on some elements. CSS2 support which included floats and positioning were not really ready.
Most existing sites still used <font> tags, but CSS was starting to be accepted. CSS let you alter text formatting on links (removing the underline was controversial to say the least) and IE4 introduced the :hover selector on a tags, allowing for mouse-over colour changes. Yes, bold-on-hover was a real thing we did!
Page layout
Layout choices are very limited. There’s no way to break an element out of the document flow (no floats, no CSS positioning), and there are only two ways to have multiple regions on a page.
First we have frames. Frames let you divide the window in to multiple separate named documents, so you could have a left frame 200px wide called ‘menu’ containing the site menu, and the rest of the document as a frame called ‘main’. <a target="main"> in the menu would cause the link to open in the main frame. This still exists as the <iframe> element, but frames are long deprecated. I won’t use frames in the main series, perhaps as an aside.
The other was the (mis)use of the <table> element. You’ll see this soon.
Fonts
Only fonts pre-installed on the user’s system could be used, no web fonts! In the early days this meant taking a guess and hoping. In the mid 90s Microsoft released their Core Fonts for the Web project, which was a bundle of fonts licensed for non-commercial use available for download. There was some traction getting them bundled on non-Windows systems, but it took a long time. Generally multiple fallbacks were provided.
Colour choices
Even though most people in 1999 had video cards that supported at least 24-bit “true colour” at 800x600 people using 16-bit “high colour” were still around. Due to how different systems mapped the colourspace it was common for colours that were distinct on one system to be the same on another.
Images
Even with a storage limit of 10MB this wasn’t the major restriction. Most of the world used dialup modems, with speeds of 56kb/s reasonably common. Even this extremely lightweight website would take around 10 seconds to download (though without custom fonts it’d probably be sub second), and the front page of English language Wikipedia would take close to two minutes.
This means decorative images had to be kept small to avoid stuttering loads - under 10kb each. SVGs were still a few years away, so we have GIF, JPEG and PNG.
Additionally browsers often only allowed a few requests in parallel (2-6 depending on your browser) so using multiple images would result in a site that slowly drew as each image downloaded in turn. Each request needed a whole new TCP connection, in New Zealand fetching from the US that would result in a minimum time of 250ms per image.
Animations
A memory surfaced during this project. In 1999 animations stopped when you scrolled the page. There often wasn’t enough video memory for off-screen rendering, so the whole content area was re-painted on every scroll action. If you dragged the scroll bar the document didn’t drag in real time, but if you pushed the down-arrow repeatedly animations would effectively stop as the timer was disabled during the re-render.
Character set
Unicode definitely exists, but it’s not in use. The ISO 8859-1 character set is generally available, though the Windows codepage 1252 is often actually used. Non-ASCII characters are rare outside of personal sites due to cross-platform and cross-language rendering problems. I’ll be sticking to ASCII characters.
I’m not going to keep up the narrative style from the last post, so from now on it’s back to first person.
In this post I’ll go through the design updates, and I’ll post a follow-up detailing the constraints that went in to this.
Lets get designing!
First, since basic CSS worked in 1999 I’ll add some CSS inside the <head> tag to make the background trendy black and text awesome white, and set the font to something sans-serif for the body and something more impactful for the headings.
I’ve also obtained some page dividing gifs from the internet archive! There aren’t many ways to divide a page up, so images like this are used to visual separate parts of a page.
Using center tags because I don’t recall using <div> tags that early. I think it might have been one of those Netscape vs IE things? I know Netscape had the <layer> tag.
Now to add some links to the future pages of the site. The pages aren’t there yet, but that’s OK because it’s under construction right? They’ll just 404 for now
Now let’s add a fancy animated star background by adding <body background=background.gif>. Yes, CSS for this worked, but what’s more 1999 than mixing styling systems?
Now we can’t really read the links using the default blue and purple. A lot of people wouldn’t be bothered but we can fix it with some more CSS! And add a hover effect for people using IE4!
Next up it’s time for some interactivity with a guestbook!
Author’s notes
Corrections to part 1
In my first post I neglected an important detail - the background colour! In the 90s - at least when using Netscape Navigator and earlier versions of Internet Explorer - the default background colour wasn’t white, it was a mid grey colour. #C0C0C0 to be precise. Why? I’m not entirely sure. I’ve fixed this.
Web browsers are amazingly backwards compatible, almost everything from 1999 still works (except <blink>). There is a problem though - window size. It’s hard to find statistics from the era, but a substantial number of visitors would have been using 800x600 monitors, with 1024x768 also being common. By comparison my 15“ laptop has an effective screen resolution of 1680x1050, and my desktop monitors are even larger.
So I’ve decided I’m going to add some CSS to cap the document width at 1000 pixels and centre it in the browser. Though to be honest this was a common thing to do in 2008 when larger screen sizes appeared and broke the layouts.
So to follow along at home with my website like it’s 1999 series you’ll need a webserver like it’s not 1999. A genuine 1999 webserver would last about 15 minutes on the modern internet.
To set up your own server you’re going to need some degree of experience with UNIX-like systems, DNS, and domain management. I’m not sure if there are many tutorials on this out there, I learnt them a long time ago.
I’m using a virtual machine on my home server, because I’m the sort of person who has a home server. But here’s what I did:
Create a virtual machine to run the server
You could use whole physical machine if you have one handy. I created a VM with 12GB of disk space, 512MB of RAM and 1 vCPU.
An ISP would be able to afford a system with more disk space than this in 1999, but a single core of my 3.5Ghz Intel core i5 4690 would far outclass anything an ISP ran in 1999, even if it is 10 years old at this point.
Install FreeBSD 14.1
Why FreeBSD? I remember it being quite common at the time, and the first UNIX-like server I had a shell account on was FreeBSD. Linux was already well established in 1999, but a lot of the people still regarded it as an upstart that still needed to prove itself. FreeBSD is actually younger than Linux by a couple of years, but its BSD heritage gave it more weight in some opinions.
Also for this series I feel being slightly foreign to most people is an advantage.
I’m not going to include a guide for the installer, the FreeBSD website has documentation.
You can also run Linux, Alpine has everything we need available.
Create a non-root user
useradd my_user. Just remember to add it to the wheel group so you can use su when you SSH in.
Install critical administration utils with pkg
Yes I could use ports, but pkg is good enough.
pkg install vim
Install Apache httpd
pkg install apache24
I’m using Apache despite there being more modern options that are probably better for almost any other use, like nginx and lighttpd. However Apache still supports cutting-edge 90s features that most other servers have removed or never supported, like server-side include processing, CGI script support, and .htaccess support for per-directory configuration overrides. These will come in handy.
Configure Apache
In /usr/local/etc/apache24 edit httpd.conf and uncomment the LoadModule lines for include_module and userdir_module. Also uncomment the line Include etc/apache24/extra/httpd-userdir.conf.
Done!
Yep, that’s pretty much it. The root site is in /usr/local/www/apache24/data.
What’s missing?
SSL
For that true 1999 experience SSL is terminated outside the web server. An ISP in 1999 wouldn’t offer SSL as standard, it used a lot of CPU and required a dedicated IP address per domain name (mostly).
If you’re following along you should use Let’s Encrypt and certbot to configure SSL for you.
Firewall setup
You should think about this, especially if you’re intending to allow other users access to your server. You probably don’t want random servers running.
Things I may add for the future:
Email hosting, SMTP and local delivery at least, probably not IMAP or POP3 unless I move it off my home server.
A database, probably PostgreSQL. I know MySQL was more favoured at the time, but I prefer Postgres.
Perl and python modules as required to run the demos I have planned.
To change from my regular ramblings about healthcare and reading one to many rose-tinted reckons about how things used to be, let’s make a website like it’s 1999!
What this is and isn’t
My aim is to show how you got your site online in 1999, but not to be historically accurate, so
I will be using modern HTML and CSS. Quirks mode belongs in the past
The server software will be modern, because if it wasn’t the server would already be compromised.
I will be using secure methods rather than accurate ones, but I’ll note where this is different.
But I will show how we made websites dynamic and the compromises made when you had limited space!
I’ll make another post on the server setup if you want to follow along. Ask nicely and I might just give you access to TheaNET.
Setting the stage
It’s 1999! You’ve just signed up for your first internet connection with your local ISP, TheaNET! Hey that’s your name as well! They’re a very modern ISP, supporting your new V92 56k modem, and you get 30 hours of connection per month for a very reasonable $29.95.
Looking through the signup pack they sent you, there’s a CD with Netscape Communicator and a bunch of trial apps, some pamphlets, and a letter with connection details on it. One part catches your eye:
All accounts come with complementary website hosting with 10MB of disk space and 100MB of traffic per month.
Not only can you browse all the sites on the internet, you can make your own. This is truely the future.
A few weeks later
How hard can it be? You’ve gone to the local bookstore and purchased a copy of “HTML Made Easy”. The first chapter tells you to open notepad, enter this text, and save it to index.html on your desktop
<html>
<head><title>Thea's Website</title></head>
<body>
<h1>Thea's Website</h1>
<p>Welcome to Thea's website!</p>
<p>This site is still under construction, please check back soon!</p>
</body>
</html>
Now to upload it. The instructions from TheaNET say
To upload to your website, use a program like WinSCP (included on the CD-ROM) or a similar sftp tool. Copy the files in to the public_html directory (if you can’t see it, you can create it). Use your dialup login and password to access your web hosting.
After some fiddling around with the copy program you get the public_html directory created and index.html copied over. Entering https://www.thea.net.nz/~thea/ in to your web browser you see your website come to life! On the internet!
It doesn’t look great, but it’s a start, it says it’s under construction so everyone knows you aren’t finished, and you’re about to read the chapters on images and text formatting.
… to be continued!
Editors Notes
scp instead of ftp
Because I want to make this something safe for others to follow I use scp for file copying. While I could set up an FTP server not only are maintained and secure FTP servers pretty rare, the protocol really doesn’t work well with modern networking. The server I’m using does not have a public IP address.
Generally ISPs did not include shell access with hosting and only allowed ftp access, which is possible with scp as well.
What’s with ~thea?
This is a user directory! In 1999 using a per-user subdomain was not very common, and on UNIX-like systems ~ is used to represent the home directories where users store their files. ~ is your home, ~thea is Thea’s home. The Apache web server continued this convention to represent each user, but served out of public_html in that user’s home directory.
This disappeared as websites got more dynamic due to cookies. Cookies are stored per-domain, so if you had http://www.thea.net.nz/admin/ which set an admin cookie, http://www.thea.net.nz/~nefarious_user/cookie_stealer.html would be able to steal it and gain admin access.
Everyone lucky enough to be prescribe progesterone in the form of oral micronised progesterone capsules has probably been told to ensure they take them on an empty stomach. This isn’t that uncommon with medications, eating can affect the way they are absorbed. Out of curiosity I wondered what the problems are, in case I forget and have a snack just before taking it.
First, a caution. Always take medication in the manner prescribed by your doctor. This information is just for curiosity.
It turns out there was a study done in 1993. Micronised progesterone for oral administration came to the market in 1980, so it’s been around a while and a lot of the studies are fairly old. It’s a well understood product.
Concomitant food ingestion increased the area under the serum P concentration versus
time curve (AUC0 to 24) and the maximum serum P concentration (Cmax) without affecting time to
maximum serum concentration (Tmax) (P < 0.05). Micronized P absorption and elimination were
first-order processes and exhibited dose-independent pharmacokinetics between 100 and 300 mg.
So to translate that - taking your progesterone with food increases your absorption of progesterone, over the 24 hour period more total progesterone was absorbed, higher blood progesterone levels were measured, but the time taken to get to the maximum progesterone level was the same as for fasting. Secondly, the absorption of progesterone scales directly with dose, it doesn’t seem to drop down or rapidly increase.
But how much more progesterone is absorbed? From the paper’s conclusion:
Absorption of micronized P was enhanced twofold in the presence of food.
So taking progesterone with food results in twice the absorption rate compared to fasting. The peak is higher but the rate of metabolism is unaffected, so the end result is a consistently higher serum progesterone level (you can look at the paper to see the graphs and numbers)
Why are we told to take progesterone on an empty stomach? I can’t find an answer. It could be to avoid the peak - nearly six times higher when taken with food - or there could be other interactions not documented in this paper. If I find out more I’ll post an update!
1
Simon JA, Robinson DE, Andrews MC, Hildebrand JR 3rd, Rocci ML Jr, Blake RE, Hodgen GD. The absorption of oral micronized progesterone: the effect of food, dose proportionality, and comparison with intramuscular progesterone. Fertil Steril. 1993 Jul;60(1):26-33. PMID: 8513955. Free full text
One thing everyone who takes oral oestrogens, whether for contraception, menopausal HRT or gender-affirming HRT, gets told by their doctor is that oral oestrogen raises the risk of blood clots - called a thromboembolism in the medical world. However we’re never really told what the risk actually is. If the rate without the medication is 1 in 100,000 per year and the increase in risk is 10% - to 1.1 in 100,000 of people on the medication - then perhaps it’s not a problem? But if increase is to 50 in 100,000 - a 500% increase if I can do maths today - then that might be more concerning.
So let’s dig in to the literature again!
Summary (for the TL;DR)
Yes oestrogen HRT increases the rate of blood clots, but dose doesn’t make much difference when using bio-identical 17β-estradiol. The difference in risk of using pills, patches gels or injections is still not clearly understood. The rates reported are not adequately controlled for dose form, oestrogen type, and additional risk factors.
Synthetic oestrogen such as ethinyl estradiol have a much higher risk than bio-identical 17β-estradiol.
Overall the risk of HRT containing bio-identical 17β-estradiol is higher that the general population, but still a very low risk.
Does oestrogen itself increase blood clots?
While a study showing a correlation between oestrogen levels during pregnancy and blood clots has previously been widely circulated, a more modern study1 shows that in the absence of oestrogen-containing medications sex hormone levels have no significant impact on rates of blood clots. Oestrogen levels during pregnancy are an order of magnitude higher than would be achieved by any form of medication.
So no, oestrogen levels are not the cause of blood clots, at least when in the normal physiological range.
The story is different with oestrogen-containing medications
Does HRT increase the rate of blood clots?
In short, yes.
The main clots that have been studied are venous thromboembolisms - VTE - which are clots that form in veins, generally in your arms and legs. We’re often told it depends on the dose level and route of administration - pills, transdermal patch/gel, injection or implant - with pills being the worst.
Fortunately there have been two studies performed on this recently! They are both meta-analyses, which means their authors haven’t done a direct study, but have identified previous good studies and combined their results. In particular they’ve found studies that not only recorded the rate of VTE, but also the type of oestrogens used, the dose, route of administration, and any other hormone medication used alongside. So many papers just say “transgender HRT has this rate of VTE” with no detail of what form, dose, or administration route is being used! This makes my job easier.
I’m only going to refer to one of the papers here, because it cites the other. I’ll link to both.
Publication: Managing the risk of venous thromboembolism in transgender adults undergoing hormone therapy 2
The paper I’m going to cite identified 13 studies that had observed the rate of VTE and also noted the form of oestrogen and dose used. I’m going to be a bit naughty and do a pull-quote from the results before discussing it more, to set the stage a bit
Finally, even if the risk from exogenous estrogen use remains significant statistically, the absolute clinical risk remains low.
In plain English - even if HRT causes an increase in blood clots which can be measured, it’s still a very low rate.
In depth
This analysis found that the synthetic oestrogen ethinyl estradiol causes a much larger increase in the rate of VTE than bio-identical estradiol. Synthetic progestins - such as cyproterone acetate - were also found to have a larger effect on VTE rate than estradiol valerate in any form.
The paper also says that:
Data suggest a positive correlation between estrogen use and VTE. A recent review3 found the overall incidence rate to be 2.3 events per 1,000 patient-years.
which cites the other paper - but there’s another point made. VTEs are associated with other health conditions and behaviours. Smoking tobacco is well known to increase the rate of blood clots, as are having high blood pressure, hypertension, undergoing surgery, and being HIV positive. Additionally acute stress and other mental health conditions are correlated with blood clot rates. From the paper:
Only one study to date demonstrates an occurrence of VTE in the absence of risk factors beyond hormone therapy.
That study4 followed 676 trans women over 8 years on predominantly oral oestrogen, and only had one incidence of VTE.
Their conclusion?
Firstly, avoid ethinyl estradiol containing medications. They have a significantly higher risk of blood clots
Secondly,
Although there seems to be clear evidence that transdermal estrogens dosed up to 0.1 mg/day or below are a lower risk for VTE than other forms of estrogen, it is unclear whether this is related to the delivery method or a dose effect.
0.1mg/day is a low transdermal dose. The current PATHA guidelines go up to 0.2mg/day, which has similar risks to oral estradiol valerate.
Risk mitigation is an important part of the care of transgender patients due to the many risks associated with not providing hormone therapy (ie, poor mental health) and the potential risks associated with hormone therapy. Further study of the relationship between estrogen and the risk of VTE will serve to inform the safest possible care for transgender patients
So as always, the conclusion is there’s not enough data to draw a solid conclusion, but treatment should be based on risk mitigation and acknowledge that focussing on risk minimisation may have worse outcomes for the patient.
So where are we?
Women not on any form of oestrogen medication have a VTE rate of around 4.2 per 10,000 person-years. With a rate of 23 per 10,000 patient-years according to that study that’s a large difference, but the analysis included use of synthetic hormones that are known to have far higher risk. They also noted that only one study identified a blood clot occurring without other know risk factors present - that paper gave a rate of 7.8 per 10,000 person-years.
Unlike menopausal HRT, the risks associated with different delivery mechanisms is not as clear-cut.
So yes, you will have a higher risk of blood clots on any form of estrogen and you should try to manage it through other factors you can control, such as not smoking and monitoring your health.
1
Holmegard HN, Nordestgaard BG, Schnohr P, Tybjaerg-Hansen A, Benn M. Endogenous sex hormones and risk of venous thromboembolism in women and men. J Thromb Haemost. 2014;12(3):297-305. doi: 10.1111/jth.12484. PMID: 24329981.
2
Goldstein Z, Khan M, Reisman T, Safer JD. Managing the risk of venous thromboembolism in transgender adults undergoing hormone therapy. J Blood Med. 2019 Jul 10;10:209-216. doi: 10.2147/JBM.S166780. PMID: 31372078; PMCID: PMC6628137.
3
Totaro M, Palazzi S, Castellini C, Parisi A, D’Amato F, Tienforti D, Baroni MG, Francavilla S, Barbonetti A. Risk of Venous Thromboembolism in Transgender People Undergoing Hormone Feminizing Therapy: A Prevalence Meta-Analysis and Meta-Regression Study. Front Endocrinol (Lausanne). 2021 Nov 9;12:741866. doi: 10.3389/fendo.2021.741866. PMID: 34880832; PMCID: PMC8647165. Full text
4
Arnold JD, Sarkodie EP, Coleman ME, Goldstein DA. Incidence of Venous Thromboembolism in Transgender Women Receiving Oral Estradiol. J Sex Med. 2016 Nov;13(11):1773-1777. doi: [10.1016/j.jsxm.2016.09.001])(https://doi.org/10.1016/j.jsxm.2016.09.001). Epub 2016 Sep 23. PMID: 27671969.