I’ve recently been forwarded a document that the Victoria University of Wellington’s Mauri Ora Student Health clinic has been using to blanket deny their patients progesterone, even going as far as to forcibly discontinue the prescription from people already prescribed it.

I normally avoid doing a critique of a specific practice because there are many factors that go in to the decision to prescribe, but I was shocked that this practice would just invalidate a standing prescription without any evidence of harm. Not prescribing because of a lack of knowledge of the correct process or side effects is one thing - a doctor cannot provide informed consent if they themselves are not informed - but to remove an existing medication because they don’t like it is quite another. Imagine if a doctor refused to continue a contraceptive prescription because they had a personal belief against it? In certain circumstances doctors can refuse to prescribe due to personal beliefs, in which case they are required to refer you to someone who will.

So lets pick apart their reasoning to see if there’s anything behind it, because if progesterone is dangerous despite multiple sources saying it should be up to the patient I’d like to know, and I’m sure a lot of others would as well.

I’ll link the full text below, OCR was used on photos sent to me to extract the text.

So let us look at their reasoning, as presented in their information sheet. I’ll use their sections as well.

Evidence

Currently, no medical evidence exists that progesterone provides any additional feminisation benefits for TGNB people

This, as I’ve previously discussed is technically correct. There is no medical evidence that progesterone provides any benefits. There is also no medical evidence that progesterone does not provide any benefits. There are two studies on the effect of progesterone in trans women I know of, both are inconclusive due to a small cohort and short duration, though there is a large scale study in the Netherlands which I think has finished so will hopefully publish soon.

However they explain this more, so lets go in to their reasoning

No high quality evidence that progesterone increases breast tissue in TGNB people.

Again there is no evidence because nobody has looked. The study I mentioned above is looking at this.

Evidence shows that most trans feminine people are able to achieve size AA- or A-cup breasts over sustained treatment with oestrogen and androgen blockage, with maximum effects achieved over 3-5 years. After this point and during treatment, complementary interventions are available […]

Read: “Because we refuse to prescribe proper levels of hormones our patients never achieve real breast development so we recommend implants, padded bras, and breast forms”.

Ok I was a tad angry there.

There is nothing to suggest that when given appropriate levels of hormone therapy trans women will not achieve the same breast development as cis women.

Anecdotally some TGNB people feel progesterone is helpful for mood or sleep. There is no evidence that progesterone plays any role in the natural sleep cycle. …

Saying progesterone does not affect sleep is simply¹ not² true³. There are many studies showing that micronised progesterone has a positive impact on sleep, and is part of the natural sleep cycle. The impact on mood is anecdotal, though high levels of progesterone correlate with a decrease in mood during the menstrual cycle.

… A deterioration in mental wellbeing is a recognised side effect of most forms of progesterone treatment, and at present there is no good quality evidence to suggest that progesterone improves mood in any way.

Actually true for once. It’s not just progesterone though, it’s all progestins that are associated with negative mood impact. Progestins are a group of hormones including progesterone, several metabolites of progesterone, and many synthetic progestin-analogs that are used in medicines.

The main anti-androgen used in New Zealand is cyproterone acetate - a progestin around 1000 times more effective at binding to progesterone receptors than progesterone itself⁴. Rejecting progesterone for this while prescribing cyproterone acetate is rather disingenuous.

Ok. Next section

Risks and negative effects

In all shared decision-making in medicine, we must balance risks and benefits for every patient.

Absolutely true. However the risks should be balanced against the patient’s ability to make informed consent for the level of risk they’re willing to tolerate, not what someone else thinks they should tolerate.

Blood clots

[…] Several factors affect your risk of clots (such as family history, smoking, recent surgery). We know that taking oestrogen (excluding ostrogen patches) increases the risk of blood clots, […]

Ok stop here. All oestrogen at GAHT levels including patches raises your risk of blood clots. Patches aren’t a get-out-of-jail free card. Again I’ve previously looked at the risk of blood clots in HRT, and the risk doesn’t increase with dose until you get to extremely high doses. There has only been one documented case of blood clots in a person receiving bio-identical gender-affirming hormones who did not have additional risk factors (smoking, diabetes, surgery, HIV etc).

The risk for trans women using hormones is about double that of a cis woman who isn’t using hormonal birth control, but that level is so low it’s not worth risking someone’s short term mental health over a very unlikely long term risk. The risk of blood clots from hormonal birth control is much higher, yet there’s no restriction on their prescription.

Newer forms of progesterone such as utrogestan have a lower risk of blood clots than older forms; but further studies are needed.

Utrogestan - micronised bio-identical progesterone - was introduced to the market by Besins Healthcare in 1980. That’s 45 years ago as I write this. There have been plenty of studies on it and they have shown that there is no blood clot risk from utrogestan⁵.

Older forms such as medroxyprogesterone acetate are synthetic progestins, utrogestan is identical to natural human progesterone. If natural human progesterone or its metabolites caused a significant increase in blood clots it would be a problem for all women.

Cancer

Animal studies have shown an association between progesterone and breast cancer. Progesterone use in older cis women beyond the menopause also appears to be associated with an increase in breast cancer risk. All TGNB people taking oestrogen with breast tissue development are already at increased risk for developing breast cancer.

OK. Lets pick apart that last point, because there’s a large study following over 2000 trans women and 1000 trans men in the Netherlands⁶ a great meta-analysis paper⁷ on this. A meta-analysis is one that looks at several existing studies to make what is effectively one larger study.

So do trans women have a higher breast cancer risk? Well yes, but also no.

Both studies say trans women have a significantly higher - the dutch study says 46 times higher - risk of breast cancer than cis men. But what about against cis woman who have a much higher rate of developing breast cancer than cis men? Both studies again agree, trans women have a lower risk of breast cancer than cis women. The Dutch study that looked at trans women with a median of 18 years on HRT, and a max of 37 years, but they controlled for the time since puberty for cis people and time since starting HRT for trans people.

The studies don’t lie - trans women do not have a higher risk of breast cancer than cis women. The risk compared to cis men is not relevant as cis women have a far higher rate of breast cancer.

• In cisgender women utrogestan does not seem to increase breast cancer risk in the first 5 years of use, but there is some increased risk beyond this.
• There are no data regarding breast cancer risk in TGNB people on progesterone.

Actually true for once, backed by a study⁸. However that study also states:

women should also be counseled that the possible increased breast cancer risk with combined MHT is small (<1 per 1000 women per year of use) and lower than the increased risks associated with common lifestyle factors such as reduced physical activity, obesity and alcohol consumption

So again this is a very small additional risk for a population who already have a lower risk of cancer than cis women.

Other risks

Progesterone use in cisgender women is associated with:

• Low mood (depression)
• Weight gain
• Low libido (low sex drive)
• Fatigue

These are associated with progestins, as mentioned cyperoterone acetate is also one. These are also symptoms that are experienced by some women taking progestins as part of hormonal contraception and are normally managed symptomatically.

As with any medication negative side effects have to be part of the informed consent discussion, and if they appear this needs to be managed symptomatically. There’s no issue with recommending discontinuation of progesterone due to side effects, but the presence of potential negative side effects is not a reason to refuse to prescribe. If that was the norm then no medicines would ever be used.

The rest of this “information” sheet goes in to the effect of progesterone on cis women. I’ve picked the most relevant points

Progesterone in cisgender women

During a cis female puberty, progesterone is involved in the development of breast ducts, which are needed for future breastfeeding, but it does not appear have a role in breast volume/size.

This is saying “Progesterone is important for breast function, but trans women will never need functioning breasts so we don’t care.”

There’s no physiological difference in breast tissue no matter how you start growing them, so why would you think a process that’s important for development in cisgender women is not important for transgender women?

When doctors induce is puberty when this has not commenced naturally, we avoid introducing progesterone until breast development is complete, as earlier use appears to blunt breast growth.

This is true, and a good argument for not introducing progesterone at the start of HRT. However it can’t be used as an argument for never introducing progesterone as it’s clearly important.

Progesterone levels are low throughout most of the menstrual cycle in cis women, and only rise transiently after ovulation.

Progesterone levels rise for approximately two weeks of the cycle from around the same level as cis men to 4-10 times that. So for half the time cis women have a significant amount of progesterone.

I haven’t looked in to the effect of testosterone suppression on progesterone production. I should, because it would be interesting to know if using a synthetic progestin like cyproterone acetate reduces the normal production of progesterone. While cyproterone acetate works like progesterone it doesn’t produce the normal range of metabolites such as allopregnanolone, which are known to have different functions.

Contraception […]

Contraception uses synthetic progestins. The side effects are well known and don’t prevent their use. The side effects are discussed as part of the informed consent process.

Menopause

Menopause is a clear set of symptoms to be treated and the level of medication can be compared to the symptoms. You can’t compare the effects of GAHT to the symptoms because the treatment has to last a lifetime.

Conclusion

We have a duty of care and professional practice to do no harm and to practice evidence-based medicine.

As has been outlined in many studies on gender-affirming treatment, the decision to take no action is not a neutral stance. Refusing to prescribe because the medication has a small chance of causing harm in the future is trading the patient’s possible future physical health against their very real short term mental health. Doctors should work with their patients and be able to explain the risks and lack of evidence of benefits to their patients. The only proven risks of progesterone are the same as hormonal contraceptives, and these are freely prescribed.

My conclusion

This so-called “information sheet” has many inaccuracies and overplays risks based on inaccurate comparisons to avoid prescribing a medication that has very few negative side effects.

The problem with the need for doctors to prescribe gender-affirming hormone therapy is they often treat it like any other medication. They want it to cure something, and with any medication you use the smallest amount needed to cure the symptoms. However being transgender cannot be cured or treated solely on a physical symptom basis. While I don’t believe that GAHT should be unregulated, I do think clinicians need to work with their patients to find a regime that they feel physically and mentally well on, and the current guidelines are far too conservative.

The only model we have for appropriate hormone levels is that of cis women, and the range is far wider than the current New Zealand GAHT guidelines acknowledge.

Citations

The problem with having to get a doctor to prescribe gender-affirming hormone therapy is that doctors are trained to treat diseases. When you’re treating a disease you have a clearly defined goal - curing the disease, or minimising the suffering if it cannot be cured. This also extends to the scientific studies used to formulate treatment options, there is a clear and measurable definition of success. This is critical for normal patient care, doctors want to minimise the harm caused - and lets be clear, all medicines are harmful, just less harmful than what they’re treating - while maximising the effectiveness of the treatment.

This doesn’t work for gender-affirming care though. The effects you get from hormones depend a lot on your genetics and we don’t know enough about those to determine if a treatment is fully effective. In terms of M-F transition, we can’t establish if a certain breast size is the one genetically encoded, or if it has been restricted due to a lack of hormones. For those going F-M, is your voice fully deepened or has it stopped because of insufficient testosterone? Does the hormone level actually matter to either or is there just a threshold? We cannot design a study for this, especially given it would have to run for over 5 years to fully capture all the changes.

All of the uncertainty leads to doctors missing out on one half of the risk calculus - if you can’t tell what the target is you’re stuck with minimising the overall risk rather than balancing it against the effectiveness of treatment.

Now I’ll focus on one of the problems this causes

As always I’m looking at academic papers, so there will be medical language used some of which may be outdated.

Arbitrary medication limits

In the latest edition of the PATHA guidelines for gender affirming care, which is the document used by most doctors in New Zealand, there is this paragraph:

There is currently no evidence to suggest that a dose of ostrogen higher than 200mcg/24 hours via patch or 6mg daily orally is helpful, and, indeed, poor evidence to suggest any strong correlation between ostrogen doses at recommended levels and outcomes at all.

which cites as evidence

Moore E, Wisniewski A, Dobs A. Endocrine treatment of transsexual people: a review of treatment regimens, outcomes, and adverse effects. The Journal of Clinical Endocrinology & Metabolism. 2003;88(8):3467-73.|

Time to get the paper! Here’s the summary of the paper on pubmed, your local library may have access to the journal if you want a copy.

At first glance, this paper suffers from very common issues in the field of transgender healthcare. It’s based on outdated medications and, the time period is unclear, some of the discussion compares to post-menopausal HRT, and it contains unjustified dose-level scaremongering.

Outdated medications

In this study the trans-feminine cohort were predominantly treated with ethinylestradiol or conjugated equine estrogens (CEE). There is one that used bio-identical oestrogen, but this was delivered as an injection for the first year.

These are not the medications we use in New Zealand. Both ethinylestradiol and CEE are no longer used for most GAHT because of the known adverse effects which are not present or greatly reduced when using bio-identical hormones, for example blood clots which aren’t a serious problem with bio-identical oestradiol.

It’s also important to note that non-human oestrogen has a different affinity for the various oestrogen receptors in the body. Two studies ^{1 2} found very different affinities for ethinylestradiol to the human oestrogen receptor, but neither of these had the same balance as human oestrogens do. Does this make a difference in GAHT? We have no way of knowing. CEE has a different binding profile as well.

Additionally most of the reports used cyproterone acetate in doses of 50-100mg per day which is a very high dose of a medication with known dose-dependant side effects, and is also known to be just as effective at much lower doses. Cyproterone acetate is a progestin, and has a very high affinity for progesterone receptors, which are implicated in breast development in animal models.

Study duration

It’s hard to pinpoint the duration of this study as it’s based on reports from multiple worldwide practices. They all seem to have different standards, but it takes several years for transition - puberty takes 8+ years and there’s nothing to indicate gender transition would be different - so studies have to run for a long time to see all the effects.

Comparison to menopausal HRT

This happens a lot in so many studies. Menopausal HRT and GAHT are not comparable in their goals and measures of efficacy. HRT for menopause has clearly defined symptoms so the dose can easily be adjusted to the lowest possible to relieve them, and lets remember that the surge of demand for HRT came after a TV documentary showed how badly treated women with menopause were by doctors, either through ignorance, not updating their knowledge about treatments, or plain dismissal of symptoms as important.

Menopausal HRT is aimed at reducing symptoms from a drop in hormones, not replacing hormones to effect a change. Doses for GAHT will have to be higher, we should be aiming for adult levels of hormones.

Dose level scaremongering

Quotes from the paper

Treatment that exceeded recommended estrogen dosages in M3F transsexual people was reported by eight subjects (45%), and five subjects (28%) reported three or more times the recommended dosage. […] Additionally, despite the hesitancy of providers to distribute injectable hormones to M2F transsexual people, 28% reported their use. In light of the older age of subjects, these high doses and complex regimens were particularly concerning for increased risk of adverse effects.

Doctors are always concerned about the risk of adverse effects, but they rarely seem concerned about the patient’s concern for these risks, or attempt to balance it against the mental distress of gender dysphoria.

All medications carry the risk of adverse effects, it should be up to the patient to determine their risk level.

My conclusions

I don’t think this study provides the evidence for any restriction of dose levels. There are reasons to had a maximum dose level, but this study doesn’t provide them. It reports on medications that we do not use, that have different absorption and biological effects, and seems more interested in commenting about how the doctors are concerned about the levels than the patients’ overall wellbeing.

While I’m not advocating for a hormone free-for-all I do wish that doctors would work with us rather than against us, ensure we know the long and short term risks and allow us to make our own choices.

When you have more than a handful of pages on your site updating anything shared between them becomes tedious. Things like your site menu, a copyright footer if you have one, perhaps the default header and meta tags.

There were options. You could use a tool like Macromedia Dreamweaver, GoLive CyberStudio, HoTMetaL, or HotDog (one of New Zealand’s .com boom success stories!) to author all your HTML and copy the files you could use their built-in templating features, or copy/paste the changes.

If you weren’t rich enough to use those tools - they did cost quite a bit - you’d either have to roll your own using local scripts, or rely on server-side features.

The most obvious is PHP. This is one of the reasons PHP was created, PHP originally stood for Personal Home Page and was a CGI script. The original PHP syntax was very different to what ended up being PHP3, in fact it was very similar to what I’m about to use.

I’ve updated my website with all the pages, common elements brought to you by Server-side Includes!

Server-side Includes

These date back to the NCSA httpd from 1993, one of the first web servers, and are still supported by modern servers like nginx. At their most basic server-side includes, or SSI, are web server feature that looks for special HTML comments and replaces them with the contents of another file.

<!--#include virtual="./header.html" -->

This tells the web server to resolve the path ./header.html and substitute it in. virtual means use the HTTP path resolution rather than filesystem paths. Most web servers prohibit using file with an absolute path or with any ../ in the path. This is because the web server often has permission to read many other user’s home directories, so it’s possible to get around file permissions with this.

virtual also allows you to reference CGI scripts. I could have implemented my counter to return text and used <!--#include virtual="~thea/cgi-bin/counter.cgi" -->.

Files included this way may also be processed for SSI themselves, resulting in several levels of include if you go that far. Most servers have a limit on how deeply nested includes can be.

Apache also allows you to configure CGI scripts to have their result processed by the SSI module, though this is not the default setting. Apache also has an additional option called XBitHack, which lets you set the execute bit on your HTML rather than renaming to .shtml. You can also configure it to process all files for include directives, but this has performance implications. Or at least it did in 1999.

Variables

Typically the query string and some request and server details are exposed as variables.

Apache’s implementation SSI has rudimentary support for setting variables, <!--#set var="title" value="About" -->. Interpolation is also available to build variables out of other variables. You can then <!--#echo var="title" --> to get the value.

SSI does not have any form of namespacing, but whether variables set in one file are visible in another depends on the server. In Apache they are, so I used variables to pass down the title. Normally included pages have access to variables set in the parent.

Conditionals

Most SSI implementations have basic conditionals - <!--#if -->, <!--#else -->. I haven’t had the need to use them, but they could be used to include different files based on browser user-agent. This was quite handy when you needed very different styles for IE and Netscape.

As many servers supplied the query string to the page you could make a slightly dynamic page using SSI conditionals and query strings. Some nefarious people used this to implement loops

Error handling

[an error occurred while processing this directive]

That’s pretty much all you get! Unless you’re the server admin and can check the logs you’re kind of stuck.

And that’s it

Server-side Includes are a very basic tool, but they can be very powerful. However we moved on to more powerful things, like the aforementioned PHP, which I may just cover soon.

As promised, I’m going to go through how the CGI scripts work. This is not a Perl how-to, or even an example of good perl, but more a description of the process of writing any CGI script.

You can download these scripts from my script archive!

I’ve also fixed the guestbook on the site!

counter.cgi

#!/usr/local/bin/perl
use CGI;
use GD;
use GDBM_File;

The first line, called the shebang, specifies the interpreter to use for the file - this is a UNIX-like feature where any file can be ‘executable’ by specifying the interpreter to launch. In my case as I’m using FreeBSD the perl interpreter is in /usr/local/bin/perl. If you’re familiar with *NIX systems you might wonder why this isn’t /usr/bin/env perl, and it’s not because FreeBSD doesn’t have env. Scripts run in the context of the web server, and the web server does not have /usr/local/bin in its PATH environment variable.

Then we import a bunch of modules we’re going to use. Modules were almost always installed by the sysadmin and you dealt with what was available. However there were a bunch commonly installed, like CGI and GDBM. GD was less common, but was often available on dedicated CGI hosts.

$datafile = '/home/thea/web_data/count.db';
$font = '/usr/local/share/fonts/oldschool-pc-fonts/Ac437_IBM_BIOS.ttf';

$cgi = CGI->new;
$page = $cgi->param('page') or '';

Set up the font and the location of the data file, which must be web server writable or in a directory that is web server writable. It has to be an absolute path otherwise it’d be relative to the web server’s home, which on most systems is created read only.

Then initialise CGI. Perl allows you to omit parenthesis in some cases so CGI->new is actually a method call to create a CGI object. Also grab the page name out of the query string using the new CGI object. Perl actually has both || and or as boolean or operators, they differ only in precedence. This can be handy sometimes, and make things more confusing for the next person.

$db = tie %hash, 'GDBM_File', $datafile, GDBM_WRCREAT, 0640
    or die "$GDBM_File::gdbm_errno";

$count = $hash{$page} or 0;
$count++;
$hash{$page} = $count;
$db->sync;
untie %hash;
$db->close;

This is where the counting happens. tie is a perl built-in that creates a hash (aka hashmap or dictionary in other languages) backed by some sort of store. In this case GDBM_File, which uses the GNU implementation of the DBM (DataBase Manager) key-value store. This is a fast store that uses file locks to synchronise access, so we pass GDBM_WRCREAT as the access mode - for Write, Read, Create if not found, in mode 0640, or user read/write, group read, everyone no access.

You may notice that hash is first declared as %hash, then later used as $hash. Unlike other languages like Javascript and PHP Perl does not just use $ to mean variable, there are five possible “sigils” to use the perl term. The ones I’ve used are $ for scalar variables, @ for array variables, and % for hash variables. The sigil indicates how to access the variable, not what type of variable it is. As seen here if you’re talking about the whole hash it’s %hash, but if you’re trying to get a single item from the hash it’s $hash. Sigils are one of the things that make perl very confusing.

Then we update the count, sync the DB, then untie and close it so another request can be processed.

$im = GD::Image->new(100,28);
$im->colorAllocate(0, 0, 0);
$white = $im->colorAllocate(255, 255, 255);
$im->stringFT($white, $font, 12, 0, 10, 20, sprintf('%06d', $count));

Next draw the count graphic using GD. GD is a graphics drawing library with bindings for multiple languages. Here we create a 100x28 pixel image, set up a black and white colour then draw the counter. sprintf is similar to the C function and here it’s used to output the text with six digits, padded with leading zeros.

The image background is the first colour allocated, so there’s no need to draw the black background.

print $cgi->header(
	-type => 'image/gif',
	-cache_control => 'no-store'
);
print $im->gif;

Finally print the CGI headers and then the GIF itself. The header call syntax is a way to provide a hash argument, it’s a bit weird but that’s Perl for you.

Discussion

I made a few choices here, firstly using DBM for storing the counts. It was common to use a comma or tab delimited file instead, or use a single file per counter. GDBM takes care of concurrency for me, so I used that.

Next in choosing GIF. GD supports many output formats, including modern ones. but I used GIF for old time’s sake. Technically back in 1999 it’d be more likely to use PNG as the patents on GIF caused support to be removed from a lot of libraries, including GD where GIF support wasn’t restored until 2004. However a lot of people used old versions of GD with GIF support remaining.

It could do more, like assign cookies and only count each person once per day, as I’m not in the EU I don’t need to worry about permission, or track data server side to try to reduce duplicate counts, but then the number would be lower! GDBM is a key-value store so it’s easy enough to add, iterate, and look up entries, but it only supports storing scalar values so we can’t store an array or hash without serialising it manually.

But there’s a web counter in under 30 lines of code!

guestbook.cgi

#!/usr/local/bin/perl
use strict;
use CGI;
use POSIX;
use Fcntl qw(:flock);
use HTML::Escape qw/escape_html/;

shebang and module imports. Here we use strict, which will throw an error if we use certain Perl constructs that are common pitfalls, and yes this is where 'use strict'; in Javascript comes from! I didn’t use this in the counter script.

HTML::Escape is a modern module but it isn’t 1999 now so I’m going to escape HTML.

my $url = '/~thea/guestbook.html';
my $file = '/home/thea/public_html/guestbook.html';
my $cgi = CGI->new;

Set up some variables, like file paths and the URL of the guestbook file. And here we see the first strict change - the my keyword. my puts the variable in lexical scope, without my the variable will be in global scope no matter where it’s declared. This is exactly like the behaviour of Javascript in non strict mode with var.

if ($cgi->request_method eq "POST") {
	my $name = escape_html $cgi->param("name");
	my $message = escape_html $cgi->param("message");
	my $website = $cgi->param('website') or '';

If the HTTP method is POST then we grab the parameters. param will fetch from either query string or post body, which is normally considered bad practice today. Additionally we run escape_html on the name and message fields, again omitting the parenthesis from the function call.

	$message =~ s/\r?\n/<br>\n/g;
	$website =~ s/[^a-zA-Z0-9\/\\:\?=& _\.-]//g;
	$website = substr($website, 0, 50);

Next we do two things, replace new lines with HTML line breaks, and remove unknown characters from the website field. I could use escape_html here, but I though I’d show a more 90s method. This demonstrates Perl’s =~ operator, which sets the var to the result of the regexp substitution on the right.

	my $time = strftime "%a %e %b %I:%M%p %Z", localtime time;

Grab the time, for the timestamp. This is the POSIX strftime function, so the formatting operators depend on your operating system.

	open(FH, '+<', $file) or die 'Cannot open guestbook file';
	flock(FH, LOCK_EX) or die 'Cannot lock file';

Open the guestbook file and take an exclusive lock on it. A wiser solution would to loop this in case of concurrency issues, but is that really going to happen?

File handles in Perl are a special type. They can be assigned to regular with my $fh, but that’s not required. Just remember that FH is in global scope.

	my @content = <FH>;
	seek $fh, 0, 0;
	truncate $fh, 0;

Read the file in to @content. This is the first time we see a Perl array, and this construct reads the whole file line-by-line in to @content. Then we seek to the start of the file and truncate it.

	for (@content) {
		if (/<!--NEXT_ENTRY-->/) {

Next loop over the file until we find the next entry marker. You may wonder where the loop variable is, and notice there’s no variable specified in the if. Perl has an implicit variable - $_ - which is automatically used in many functions if you don’t specify a variable. So in this case $_ is the loop variable and the if automatically test against this. $_ is addressable as a regular variable as well.

			print FH "<p>$message</p>\n";
			if ($website) {
				print FH "<p><a href=\"$website\">$name</a> at $time</p>\n";
			} else {
				print FH "<p>$name at $time</p>\n";
			}
			print FH "<hr>\n\n"

This just prints the new entry in to the guestbook

		}
		print FH;
	}

	close FH;

And finally print the current line with the implied $_ argument, the close the file handle. print to a filehandle doesn’t automatically use $_ if you use a variable-bound filehandle.

}

print $cgi->header( -status => 302,  -location => $url );
print "Redirecting...";

Finally redirect back to the guestbook page! Under 50 lines of code!

Discussion

This method of directly modifying the file was quite common. Given most of the time you couldn’t access the server other than via FTP this allowed you to moderate the guestbook by downloading, editing, and re-uploading.

It was also pretty normal to ask for and publish an email address, but then spam stopped this practice.

So what next?

Well this was my foray in to CGI scripts! Next up some other 90s classics are up, like a webring!

I find it interesting how Javascript and Perl both had global by default variables and use strict to prevent those footguns, but somehow Javascript ended up with multiple null types (Perl doesn’t have null, only undef for undefined) and automatic semicolon insertion. Perl has many more features to make incomprehensible code though.

The next instalment in building a website like it’s 1999 - adding some interactivity! After all if people can’t submit things on your website how else will you hear from them?

I’m going to add two critical parts of a 90s website - a hit counter and a guestbook.

The counter is on the homepage, as it should be.

The guestbook page now works! It has a completely different style from the homepage, as was the fashion at the time. You have to show off all your design skills.

Running code on the server

In 1999 you couldn’t just run code on the server. Most hosts only allowed you FTP access to upload files, there was no way to start a process. The two most common ways to run server-side code were PHP and CGI. PHP3 had been around for a couple of years and had some features that made it really suited for shared web hosting, but I’m not going to use PHP (yet).

Shared hosts would move from allowing CGI to only allowing PHP for some of the reasons I’ll explore. PHP’s big selling point was how well it integrated with HTML, and being less annoying than CGI.

CGI

Common Gateway Interface, CGI, is a specification for web servers to execute programs and pass in details about the request. The program will then output an HTTP response with a few special headers, and the response is forwarded on to the client.

CGI programs could be anything the host system could execute, they can be compiled programs but since most of the time shared hosts did not allow you to compile code various interpreted languages were used. The most common was Perl, even though there were widely available alternatives like Python. Perl was commonly used by sysadmins and therefore usually available, had many modules installed, and fast. Having modules installed was important, your disk space was limited so you’d want as much stuff installed at the system level as possible.

The big gotchas

The biggest gotcha is CGI scripts run as the webserver’s user. Most providers did not give you access to a database, so all data had to be stored in files in your user directory. This has two big implications - first that you have to give the webserver permission to write files, which is accomplished by setting the file or directory’s group to the webserver’s group (www in my case), and secondly that any other user’s scripts can write to those files as well.

There are solutions to this. One is setting the setuid bit on the file which makes it execute using the file’s owner uid instead of the caller’s, but this requires remembering to do it every time. Another is using CGI wrappers that run as root and change to the appropriate user before executing the script, but those have potential security vulnerabilities.

Thea’s Script Archive

Most people didn’t write their own scripts. We got them from Matt’s Script Archive, which is amazingly still online and still serving the old scripts!

I didn’t use those scripts. Matt’s scripts are from a different time, where the answer to “You shouldn’t put the destination email address in a hidden form field, someone might change it and be able to send email to anyone!” was “Who would want to do that?”. They often deliberately do things we’d consider a security vulnerability today, like trusting user input (formmail.pl was well known as a spammer’s dream for this) and not sanitising HTML. They also use programs which aren’t available anymore for the same reasons.

So introducing Thea’s Script Archive. They have fewer vulnerabilities! (fewer security vulnerabilities not guaranteed).

The Scripts

While these aren’t genuine 1999 scripts, they have the same base principles. I’ll break down the code behind these scripts in a second post.

Counter

This is a simple hit counter. Every time it’s loaded the count is incremented, there’s no attempt to ignore bots, indexers, or repeated loads by the same person. These features would start to be added to counter, and for people who couldn’t use CGI scripts third party services existed to supply these. “Real” analytics were normally performed on the server access log files using programs like (Sawmill)[http://www.sawmill.net], which was still in business until 2021.

The image is created on the fly, there’s no caching here!

Guestbook

This is even simpler than the counter. Like many similar scripts from its time it doesn’t even use a database - if you only have FTP access to the server you can’t update a database to remove spam! Instead it simply appends to the HTML file using a specific comment to locate where to add the entry.

Some scripts allowed you to use a template hidden in the comment, but this one is very simple. HTML is escaped though.

Yesterday’s post was getting a bit long, so for those who weren’t doing web design in the late 90s/early 2000s here’s some background.

The constraints of 1999

The late 90s were a time of rapid change in the world wide web, the first big browser war was brewing. Netscape Navigator browser got bloated with the Communicator product they were pushing and Microsoft’s bundling of Internet Explorer with Windows rapidly pushed Netscape out of the market. Both implemented parts of HTML4 and CSS 1 and 2 in differently buggy ways, and they both had different ways of using JavaScript to animate elements on the page. Back then we called it Dynamic HTML!

The main constraints I’m following are:

CSS

CSS! Yes that’s right, we had CSS in 1999! CSS 1 support was reasonable, text and element alignment, text and background colour, and font face were reliable. Padding and margin worked on some elements. CSS2 support which included floats and positioning were not really ready.

Most existing sites still used <font> tags, but CSS was starting to be accepted. CSS let you alter text formatting on links (removing the underline was controversial to say the least) and IE4 introduced the :hover selector on a tags, allowing for mouse-over colour changes. Yes, bold-on-hover was a real thing we did!

Page layout

Layout choices are very limited. There’s no way to break an element out of the document flow (no floats, no CSS positioning), and there are only two ways to have multiple regions on a page.

First we have frames. Frames let you divide the window in to multiple separate named documents, so you could have a left frame 200px wide called ‘menu’ containing the site menu, and the rest of the document as a frame called ‘main’. <a target="main"> in the menu would cause the link to open in the main frame. This still exists as the <iframe> element, but frames are long deprecated. I won’t use frames in the main series, perhaps as an aside.

The other was the (mis)use of the <table> element. You’ll see this soon.

Fonts

Only fonts pre-installed on the user’s system could be used, no web fonts! In the early days this meant taking a guess and hoping. In the mid 90s Microsoft released their Core Fonts for the Web project, which was a bundle of fonts licensed for non-commercial use available for download. There was some traction getting them bundled on non-Windows systems, but it took a long time. Generally multiple fallbacks were provided.

Colour choices

Even though most people in 1999 had video cards that supported at least 24-bit “true colour” at 800x600 people using 16-bit “high colour” were still around. Due to how different systems mapped the colourspace it was common for colours that were distinct on one system to be the same on another.

Images

Even with a storage limit of 10MB this wasn’t the major restriction. Most of the world used dialup modems, with speeds of 56kb/s reasonably common. Even this extremely lightweight website would take around 10 seconds to download (though without custom fonts it’d probably be sub second), and the front page of English language Wikipedia would take close to two minutes.

This means decorative images had to be kept small to avoid stuttering loads - under 10kb each. SVGs were still a few years away, so we have GIF, JPEG and PNG.

Additionally browsers often only allowed a few requests in parallel (2-6 depending on your browser) so using multiple images would result in a site that slowly drew as each image downloaded in turn. Each request needed a whole new TCP connection, in New Zealand fetching from the US that would result in a minimum time of 250ms per image.

Animations

A memory surfaced during this project. In 1999 animations stopped when you scrolled the page. There often wasn’t enough video memory for off-screen rendering, so the whole content area was re-painted on every scroll action. If you dragged the scroll bar the document didn’t drag in real time, but if you pushed the down-arrow repeatedly animations would effectively stop as the timer was disabled during the re-render.

Character set

Unicode definitely exists, but it’s not in use. The ISO 8859-1 character set is generally available, though the Windows codepage 1252 is often actually used. Non-ASCII characters are rare outside of personal sites due to cross-platform and cross-language rendering problems. I’ll be sticking to ASCII characters.

I’m not going to keep up the narrative style from the last post, so from now on it’s back to first person.

In this post I’ll go through the design updates, and I’ll post a follow-up detailing the constraints that went in to this.

Lets get designing!

First, since basic CSS worked in 1999 I’ll add some CSS inside the <head> tag to make the background trendy black and text awesome white, and set the font to something sans-serif for the body and something more impactful for the headings.

<style type=text/css>
body { 
    background-color: black; 
    color: white;
    font-family: "Verdana", "Helvetica", sans-serif;
}
h1, h2, h3, h4, h5, h6 {
    font-family: "Impact", "Helvetica", sans-serif
}
</style>

I’ve also obtained some page dividing gifs from the internet archive! There aren’t many ways to divide a page up, so images like this are used to visual separate parts of a page.

<center><img src=zdivider.gif width=600 height=1></center>

Using center tags because I don’t recall using <div> tags that early. I think it might have been one of those Netscape vs IE things? I know Netscape had the <layer> tag.

Now to add some links to the future pages of the site. The pages aren’t there yet, but that’s OK because it’s under construction right? They’ll just 404 for now

<center>
  <a href=about.html>About Me</a> - 
  <a href=guestbook.html>Guestbook</a> - 
  <a href=pets.html>Pets</a></center>

Though I think it’s important to add another indication that the site is under construction

<center>
  <img src=underconstruction.gif width=567 height=36 alt="under construction">
</center>

Now let’s add a fancy animated star background by adding <body background=background.gif>. Yes, CSS for this worked, but what’s more 1999 than mixing styling systems?

Now we can’t really read the links using the default blue and purple. A lot of people wouldn’t be bothered but we can fix it with some more CSS! And add a hover effect for people using IE4!

a:link { color: #44F }
a:visited { color: #F4F }
a:active, a:hover { color: #F66; font-weight: bold }

And as a bonus I’ll add a picture of my cat. The photo isn’t from 1999, but the cat was alive then!

Check out the updates!

Next time

Next up it’s time for some interactivity with a guestbook!

Author’s notes

Corrections to part 1

In my first post I neglected an important detail - the background colour! In the 90s - at least when using Netscape Navigator and earlier versions of Internet Explorer - the default background colour wasn’t white, it was a mid grey colour. #C0C0C0 to be precise. Why? I’m not entirely sure. I’ve fixed this.

The site from part one has been archived at https://thea.net.nz/~thea/part1/

Styling and layout

Web browsers are amazingly backwards compatible, almost everything from 1999 still works (except <blink>). There is a problem though - window size. It’s hard to find statistics from the era, but a substantial number of visitors would have been using 800x600 monitors, with 1024x768 also being common. By comparison my 15“ laptop has an effective screen resolution of 1680x1050, and my desktop monitors are even larger.

So I’ve decided I’m going to add some CSS to cap the document width at 1000 pixels and centre it in the browser. Though to be honest this was a common thing to do in 2008 when larger screen sizes appeared and broke the layouts.

To change from my regular ramblings about healthcare and reading one to many rose-tinted reckons about how things used to be, let’s make a website like it’s 1999!

What this is and isn’t

My aim is to show how you got your site online in 1999, but not to be historically accurate, so

• I will be using modern HTML and CSS. Quirks mode belongs in the past
• The server software will be modern, because if it wasn’t the server would already be compromised.
• I will be using secure methods rather than accurate ones, but I’ll note where this is different.

But I will show how we made websites dynamic and the compromises made when you had limited space!

I’ll make another post on the server setup if you want to follow along. Ask nicely and I might just give you access to TheaNET.

Setting the stage

It’s 1999! You’ve just signed up for your first internet connection with your local ISP, TheaNET! Hey that’s your name as well! They’re a very modern ISP, supporting your new V92 56k modem, and you get 30 hours of connection per month for a very reasonable $29.95.

Looking through the signup pack they sent you, there’s a CD with Netscape Communicator and a bunch of trial apps, some pamphlets, and a letter with connection details on it. One part catches your eye:

All accounts come with complementary website hosting with 10MB of disk space and 100MB of traffic per month.

Not only can you browse all the sites on the internet, you can make your own. This is truely the future.

A few weeks later

How hard can it be? You’ve gone to the local bookstore and purchased a copy of “HTML Made Easy”. The first chapter tells you to open notepad, enter this text, and save it to index.html on your desktop

<html>
<head><title>Thea's Website</title></head>
<body>
<h1>Thea's Website</h1>
<p>Welcome to Thea's website!</p>
<p>This site is still under construction, please check back soon!</p>
</body>
</html>

Now to upload it. The instructions from TheaNET say

To upload to your website, use a program like WinSCP (included on the CD-ROM) or a similar sftp tool. Copy the files in to the public_html directory (if you can’t see it, you can create it). Use your dialup login and password to access your web hosting.

After some fiddling around with the copy program you get the public_html directory created and index.html copied over. Entering https://www.thea.net.nz/~thea/ in to your web browser you see your website come to life! On the internet!

It doesn’t look great, but it’s a start, it says it’s under construction so everyone knows you aren’t finished, and you’re about to read the chapters on images and text formatting.

… to be continued!

Editors Notes

scp instead of ftp

Because I want to make this something safe for others to follow I use scp for file copying. While I could set up an FTP server not only are maintained and secure FTP servers pretty rare, the protocol really doesn’t work well with modern networking. The server I’m using does not have a public IP address.

Generally ISPs did not include shell access with hosting and only allowed ftp access, which is possible with scp as well.

What’s with ~thea?

This is a user directory! In 1999 using a per-user subdomain was not very common, and on UNIX-like systems ~ is used to represent the home directories where users store their files. ~ is your home, ~thea is Thea’s home. The Apache web server continued this convention to represent each user, but served out of public_html in that user’s home directory.

This disappeared as websites got more dynamic due to cookies. Cookies are stored per-domain, so if you had http://www.thea.net.nz/admin/ which set an admin cookie, http://www.thea.net.nz/~nefarious_user/cookie_stealer.html would be able to steal it and gain admin access.

This will come up again later.

You forgot the doctype!

Nobody used a doctype in 1999!

So to follow along at home with my website like it’s 1999 series you’ll need a webserver like it’s not 1999. A genuine 1999 webserver would last about 15 minutes on the modern internet.

To set up your own server you’re going to need some degree of experience with UNIX-like systems, DNS, and domain management. I’m not sure if there are many tutorials on this out there, I learnt them a long time ago.

I’m using a virtual machine on my home server, because I’m the sort of person who has a home server. But here’s what I did:

Create a virtual machine to run the server

You could use whole physical machine if you have one handy. I created a VM with 12GB of disk space, 512MB of RAM and 1 vCPU.

An ISP would be able to afford a system with more disk space than this in 1999, but a single core of my 3.5Ghz Intel core i5 4690 would far outclass anything an ISP ran in 1999, even if it is 10 years old at this point.

Install FreeBSD 14.1

Why FreeBSD? I remember it being quite common at the time, and the first UNIX-like server I had a shell account on was FreeBSD. Linux was already well established in 1999, but a lot of the people still regarded it as an upstart that still needed to prove itself. FreeBSD is actually younger than Linux by a couple of years, but its BSD heritage gave it more weight in some opinions.

Also for this series I feel being slightly foreign to most people is an advantage.

I’m not going to include a guide for the installer, the FreeBSD website has documentation.

You can also run Linux, Alpine has everything we need available.

Create a non-root user

useradd my_user. Just remember to add it to the wheel group so you can use su when you SSH in.

Install critical administration utils with pkg

Yes I could use ports, but pkg is good enough.

pkg install vim

Install Apache httpd

pkg install apache24

I’m using Apache despite there being more modern options that are probably better for almost any other use, like nginx and lighttpd. However Apache still supports cutting-edge 90s features that most other servers have removed or never supported, like server-side include processing, CGI script support, and .htaccess support for per-directory configuration overrides. These will come in handy.

Configure Apache

In /usr/local/etc/apache24 edit httpd.conf and uncomment the LoadModule lines for include_module and userdir_module. Also uncomment the line Include etc/apache24/extra/httpd-userdir.conf.

Done!

Yep, that’s pretty much it. The root site is in /usr/local/www/apache24/data.

What’s missing?

SSL

For that true 1999 experience SSL is terminated outside the web server. An ISP in 1999 wouldn’t offer SSL as standard, it used a lot of CPU and required a dedicated IP address per domain name (mostly).

If you’re following along you should use Let’s Encrypt and certbot to configure SSL for you.

Firewall setup

You should think about this, especially if you’re intending to allow other users access to your server. You probably don’t want random servers running.

Things I may add for the future:

• Email hosting, SMTP and local delivery at least, probably not IMAP or POP3 unless I move it off my home server.
• A database, probably PostgreSQL. I know MySQL was more favoured at the time, but I prefer Postgres.
• Perl and python modules as required to run the demos I have planned.

Everyone lucky enough to be prescribe progesterone in the form of oral micronised progesterone capsules has probably been told to ensure they take them on an empty stomach. This isn’t that uncommon with medications, eating can affect the way they are absorbed. Out of curiosity I wondered what the problems are, in case I forget and have a snack just before taking it.

First, a caution. Always take medication in the manner prescribed by your doctor. This information is just for curiosity.

It turns out there was a study done in 1993. Micronised progesterone for oral administration came to the market in 1980, so it’s been around a while and a lot of the studies are fairly old. It’s a well understood product.

The study¹ handily titled “The absorption of oral micronized progesterone: the effect of food, dose proportionality, and comparison with intramuscular progesterone” was performed on 15 post-menopausal women, so it’s a very small sample size. To quote their results:

Concomitant food ingestion increased the area under the serum P concentration versus time curve (AUC_{0 to 24}) and the maximum serum P concentration (C_max) without affecting time to maximum serum concentration (T_max) (P < 0.05). Micronized P absorption and elimination were first-order processes and exhibited dose-independent pharmacokinetics between 100 and 300 mg.

So to translate that - taking your progesterone with food increases your absorption of progesterone, over the 24 hour period more total progesterone was absorbed, higher blood progesterone levels were measured, but the time taken to get to the maximum progesterone level was the same as for fasting. Secondly, the absorption of progesterone scales directly with dose, it doesn’t seem to drop down or rapidly increase.

But how much more progesterone is absorbed? From the paper’s conclusion:

Absorption of micronized P was enhanced twofold in the presence of food.

So taking progesterone with food results in twice the absorption rate compared to fasting. The peak is higher but the rate of metabolism is unaffected, so the end result is a consistently higher serum progesterone level (you can look at the paper to see the graphs and numbers)

Why are we told to take progesterone on an empty stomach? I can’t find an answer. It could be to avoid the peak - nearly six times higher when taken with food - or there could be other interactions not documented in this paper. If I find out more I’ll post an update!