From late 1995 through early 1999, I operated a small World Wide Web service provider named Dragonfire Internet Services. In days when personal megabit-class connections were the exclusive province of college students and 10 megabytes of file space was considered huge, I endeavored to, in my own little way, help people take advantage of the new communications medium known as the World Wide Web. While I don’t dare claim any part in how the web grew during that time, my experiences may give a glimpse into those formative years of the present Internet, so I have set down, as best I can recall, the history of Dragonfire from my point of view. I hope that the reader finds this retrospective as interesting as the experiences themselves were for me.
|1994: Dreams of Flight|
|1995: Cracking the Shell|
|1996: Care and Nurture of a Dragon|
|1997: Thrust into the Real World|
|1998: Growing Up|
|1999: Departing the Nest|
|2000 and Beyond: Final Thoughts|
While Dragonfire did not come into existence as a provider until late 1995, its story properly begins nearly one year earlier, on December 24, 1994. This was the day I obtained my first semi-permanent Internet connection, via my high school’s dial-in lines.
To provide some background for those unfamiliar with the technology of the day, personal Internet connections in the early 1990s—for those few who had them—were made almost exclusively over standard, land-based telephone lines, using devices known as “modems” to convert data to sound and back again. A typical modem of the time operated at 2400 bits per second (usually called “2400 baud”, though this usage of “baud” is technically incorrect1); since each 8-bit byte was accompanied by two control bits, data transfer was limited to a maximum of 240 bytes per second. At this rate, for example, one could watch E-mail text appear line by line on BBSes or similar text-based systems. Transferring a five-megabyte audio recording, a file size which itself was gigantic for the time, would have taken nearly six hours!
By 1994, higher-speed modems had begun to percolate down from research laboratories into the commercial world, and late that year I finally obtained my own 19.2-kilobit modem. The speed difference was amazing; for a 2008 equivalent, imagine switching from an unstable ADSL connection to a dedicated fiber-optic link. Not only were file transfers shortened by an order of magnitude, but that speed also allowed me to work on a remote computer almost as easily as on my own system. The display was not instantaneous, but reading mail or editing programs was much more tolerable than it had been; at least I no longer had to wait nearly ten seconds just to view the next page of a file.
The use of a telephone line imposed one other restriction: No voice calls could be made or received on the line while the modem was in use. For a child like myself without the means to obtain his own line, this was a critical problem: I had to get my parents’ permission whenever I wanted to connect to the Internet, and I had to limit my connection time to a minimum. (From a parent’s point of view, on the other hand, I’m sure that made it much easier to keep tabs on the child’s activities compared to today, when permanent connections are the norm.) As it happened, though, we had a second phone line installed for the small business my mother ran out of our house. She had started to wind the business down, so I became able to use that phone line for my own purposes, first on evenings and weekends, later 24 hours a day—though I did end up paying for the line out of my allowance.
Finally, there was the issue of how to connect. My high school—Montgomery Blair High School, in Silver Spring, Maryland, USA—had (and presumably still has) a strong computer science curriculum, in which I was enrolled. The school provided all of us in the curriculum an account on the school’s multiuser system, a MicroVAX at the time; in addition to accessing the system from the school’s computer lab, we could also use modems to connect from home. That served well enough for reading mail and Usenet news, but it was not enough to run a server.
At Blair, most of the system administration and management tasks were handled by computer science students rather than by staff members. In my senior year of high school, which started in September 1994, I was given the task of managing the MicroVAX system. While I had already learned a bit about multiuser system administration from conversations with classmates—some of whom were experimenting with a new operating system called “Linux”—and from my own studies, this was my first direct experience with system administration. It gave me a solid grounding in many aspects of administration that would later prove useful: managing system resources, backing up data (onto the funny little square tape cassettes the VAX used), and learning how to communicate with users who didn’t necessarily understand why the system worked the way it did. That year also provided me with a rather embarrassing lesson on the darker side of the Internet, or perhaps just a demonstration of my own naiveté, as I was completely taken in by the “Good Times” virus hoax that spread late in 1994; I did not learn the truth of the matter until after I had already warned the entire school about the dangers of the supposed virus.
In any case, one of the perks of being a student system administrator was privileged access to the school’s Internet connection via modem. Rather than only being able to log into the VAX, I was allowed to set up a SLIP2 connection from my computer at home to the school’s terminal server, gaining my own IP address. With an IP address, I could connect to any computer on the Internet without having to go through the VAX, but more importantly, other Internet users would be able to connect to my computer as well.
With those three factors covered—and, of course, with my own computer: a Commodore Amiga 2000, of which I was quite fond for both its ease of use and its programmer-friendliness—the stage was set for the appearance of what would later become Dragonfire. And on Christmas Eve, 1994, a server known to absolutely nobody as dragon.mbhs.edu made its Internet debut.
Being a bit of a computer music fan, I had amassed a fair collection of music files (modules, or MODs, as they were often called) from various FTP sites. At the suggestion of a friend, and following the Internet principle of sharing, I set up an FTP server on my Amiga, making those files available to anyone who happened across the server.
At the time, I certainly had no intention of turning my computer into a web service provider. I was aware of the World Wide Web, and it was useful enough every once in a while, but there wasn’t really that much to see, after all. About the one thing that piqued my curiosity at the time was the appearance of a company called Mosaic Communications, which released a new browser called Mosaic Netscape3. Everybody knew Mosaic, of course; Mosaic was the browser. But what was a “netscape”? If nothing else, the question evoked a number of fascinating drawings on the computer lab whiteboard.
In fact, setting the web aside, I didn’t really have any long-term plans for the server at all. At the time, it was just a way to play around with the Internet, to learn more about computers—and to make myself (and my computer) useful. That last facet in particular would, just a year later, end up leading me places I had never imagined.
As I entered the second semester of my final year in high school, we began making plans to replace the outdated MicroVAX with a new multi-user machine, a PC (a powerful 133 MHz Pentium, if memory serves) running Linux. This meant months of planning, creating procedures and scripts for transferring user data between machines, and the like. Looking back on mail I saved from that period, I must concede that, even compared to the other student system administrators, I had a rather cavalier attitude toward the whole thing. My recollection is that I managed the VAX itself well enough, but I didn’t take a very active part in any other aspects of managing the school’s systems, chiming in only occasionally with comments or program fragments I thought might be useful—though more often than not they only served to illustrate the limited extent of my own knowledge and point of view. I may have been less objective about it at the time, but given my behavior during those days, I can’t help but be amazed that such a teenager ended up administering a web service provider just a year later.
At the same time, I continued running my anonymous FTP server, arguing to the school’s computer staff that it was an educational use of the school’s Internet connection. Which it was, certainly; running a server on one’s own, and making it available to the Internet at large, is a much different issue than merely helping administer a server on an effectively closed network. Nonetheless, that claim was undeniably an excuse to pursue my own interests in spite of school rules; I can only very belatedly apologize for whatever trouble I may have caused the staff, as well as for monopolizing one of the limited number of phone lines available for connecting to the school. (I also unwittingly created the potential to consume a full third of the school’s Internet bandwidth, only 56kbps at the time, though my actual traffic never grew that high.)
June rolled around, bringing my high school years to a close. The switch from the old VAX to the new Linux system went off reasonably smoothly, and we shut the VAX down for the last time, ending my first experience with multiuser system administration. I had already been accepted to Carnegie Mellon University, and I settled back to enjoy my last two months before entering college. Somehow or other, I managed to secure continued Internet access through my high school until I left for Carnegie Mellon in late August.
July brought my first experience with trouble I had to rely on others to fix: Early in the month, the phone line I used to connect to the Internet suddenly stopped working, leaving my computer isolated. A call to the telephone company and a couple of weeks later, the problem—a short circuit in the line—had been rectified; but for a seventeen-year-old like myself, especially one who’d been spoiled by an always-on Internet connection, two weeks was practically forever. I wheedled permission out of my parents to use our primary phone line at night, and managed to get my server online for about eight hours a day, worrying all the time about people I was presumably disappointing with the downtime. (Or so I recall, anyway. It’s quite possible I had just made that up as an excuse to cover the “shock of withdrawal” from the Internet.)
At last it was August, and the time came to move out to college. I packed up my Amiga along with various other belongings, and headed out for my first taste of independent life. I was of course nervous, as I imagine most new students are, about being on my own for several months at a time; but I was also thinking ahead to bringing my server back online via the campus network. I had already learned that the entire campus, including the student dormitories, was wired for 10Mbps Ethernet, and I couldn’t wait to try out such a blazingly fast connection.
Once at Carnegie Mellon, I did of course participate in the week-long introductory activities for new students, but I probably spent just as much time setting up and working on my computer, initially connecting via modem until my Ethernet connection was configured. The university had its own computer network, known as the Andrew system—named after the university’s founders, Andrew Carnegie and Andrew Mellon; no relation to me, of course—which I began to explore as well.
It took about two weeks for my Ethernet connection request to be processed, and on September 6th, I finally had my very own 10Mbps Internet connection. I’d already had a taste of the speed from using computers in the various labs around campus, but being able to take advantage of it from my own dorm room was nothing short of incredible. (In point of fact, I couldn’t take advantage of that speed; my Amiga was simply too slow for the display to keep up.) I promptly announced the reconnection of my server on relevant newsgroups, and FTP transfers started up immediately.
Around this same time, a friend of mine—Andrew Vestal, who later went on to found a highly regarded video game news site, the Gaming Intelligence Agency—had run into a problem: His website, a source of information for videogames from the company Squaresoft (now Square Enix Co., Ltd.), had hit the 10-megabyte limit of his Internet service provider and was still growing. We’d been talking about this the previous month, while I was still on vacation; since I knew that I’d be getting high-speed access at college, and since I had plenty of disk space available, I had blithely offered to host his site on my server. After all, I figured, no big deal; my computer’s already going to be serving anonymous FTP data around the clock, so how much more trouble could a web server possibly be?
As I later told a user who inquired about the history of Dragonfire: Was I ever wrong.
While it was all well and good to offer to help out a friend in need, that plan was nearly derailed by the lack of a decent HTTP server for the Amiga. (For that matter, there wasn’t much networking software for the Amiga at all; Commodore had already gone bankrupt by that time, and the platform was clearly on its way out.) But as a computer geek, and one with a decent knowledge of programming, the solution was obvious: What else to do but build my own? So I spent most of August and much of my free time during my first two weeks at college building a lightweight HTTP server4, and by pure coincidence, I got it working the evening before my Ethernet connection was activated.
With my preparations complete—or so I thought—we began the transfer of the website data. Five days later, when we were satisfied everything was in place, the site officially went live, and the original site was replaced by a link to my server. This had two immediate effects: Activity on the server shot up by an order of magnitude or more, and I was given a harsh introduction to the perils of multithreaded programming.
Unlike modern operating systems, the Amiga’s OS had no concept of virtual memory, instead running all programs in the same physical memory space. This limitation was at least partly dictated by the lack of any memory management hardware on the Motorola 68000 CPU used in early Amiga models, though add-on programs usually billed as debugging aids provided this feature on later processors. Among other problems, this lack of memory management meant that the OS could not dynamically expand a program’s stack, as modern systems do; the stack had to be preset to a sufficiently large value, or the program would overwrite other parts of memory, usually crashing the whole system. As a result, I had acquired a habit of declaring large buffers on the program’s data heap, rather than on the stack.
In order to maximize performance, I had used a multithreaded model for my HTTP server, creating a new task (a thread, in modern terms) for each connection to the server. The task parsed the client’s request, located the appropriate file, and read it into a memory buffer—allocated on the heap, as was my habit—one piece at a time to send to the client. In most circumstances, this works fine . . . but what if two clients connect at the same time? Naturally, both tasks will read data into the same buffer—with the result that the clients end up getting random pieces of both files.
As more and more users found the site’s new location and traffic continued to grow, the “scrambled pages” problem became a major issue, and I spent the next week looking for, then (after a sufficient amount of cursing at my carelessness) fixing the bug. School had to take priority, of course, so the fix took longer than I would have liked; in any case, it was quite a busy week.
With that finally out of the way, and with classes safely in progress, I finally had a chance to relax a bit. That lasted all of three days, until the ever-growing traffic load triggered what was presumably a bug in the OS, corrupting the filesystem on the hard disk. So I switched back to panic mode, working like mad to repair the system and restore regular service. I don’t recall for certain, but I suspect my roommate may have complained about my late-night work once or twice around this time.
One more event, seemingly innocuous, occurred around this time: A visitor to Andrew Vestal’s site sent me mail, asking if he could put his low-traffic homepage on my server as well. Naturally, thinking only of trying to help others, I acquiesced.
Sometimes I can’t help but think back on this hectic September and wonder if perhaps I should have taken things a little easier, become a little more involved in college activities. Knowing my own personality, I doubt it would have made much difference in the end; I suppose it’s only human nature to ponder “what-ifs”. In any case, though, that one innocuous message—even more than the monstrous site that loomed over my dreadfully underpowered Amiga—is the spark that eventually became Dragonfire.
Having recognized the need for a more capable machine, I promptly sought out a new computer to take over server functionality from the Amiga. Being a poor college student, I naturally had no budget for a top-of-the-line computer, and I settled for an IBM PC-compatible (as we still called them in those days) 80486-based system that my father was able to secure for me. When it arrived in mid-October, I quickly dug into the installation of Linux he had preloaded on the machine, and before long I had transferred all of the web and FTP data to the new machine. The difference was noticeable, with much smoother response and less hard disk noise—plus I could once again use my Amiga as I liked without web and FTP service slowing it down.
The downside to this was that I had only one Ethernet connection; since that connection was now used by the Linux box, and since the monitor I used with my Amiga couldn’t handle the VGA display output from the PC, I had to find another way to connect my Amiga to the Internet. So I connected my Amiga to the PC using a serial cable; while the speed was of course limited, I was once again able to browse the Internet freely.
For one reason or another, I found myself with a need several days later to connect to the college’s network directly. Since my Amiga’s serial port was already being used to connect to the Linux box, I had to switch cables and reconnect the modem. Having owned the Amiga for nearly four years, I was of course familiar with the layout of the machine’s back panel, so I reached around the monitor, pulled out one cable, and attempted to plug the second into the same place. Unfortunately, I missed by the width of a couple of pins; as I shifted the plug around, trying to align plug and socket by touch, the machine suddenly turned itself off.
I think my initial assumption was that I had accidentally bumped the power switch, knocked the power cord out, or something along those lines. It wasn’t until I tried to power-cycle the computer—and it switched itself off again half a second later—that I started to get that classic sinking feeling in my stomach. I never did investigate in detail exactly where the damage occurred, and I was able to borrow another Amiga from a friend somewhat later on; but that incident forced me, however unwillingly, to jump into the PC world and learn how to make do with Linux.
While I may not have found it particularly convenient as an ordinary desktop machine, I had been looking forward to Linux for a different reason. As mentioned above, I had granted access to a second user to upload his own homepage, along with one or two more people who sent me similar requests over the following month. However, the Amiga’s OS was a single-user one, with no provision for protecting one user’s files from another; even setting aside the case of Andrew Vestal, whom I trusted implicitly, I had no guarantee that these people would not try to destroy each other’s files—or my own, for that matter. This obviously was not an ideal situation, and I looked to the multiuser capabilities of Linux to rectify it. Indeed, Linux allowed me to configure system security easily, ensuring that users’ files and the system itself would be safe from any accidental or malicious destruction.
With that problem settled, I thought back on the requests I’d received to host other users’ data. Now that I had a reasonably secure system, and one that was more than powerful enough to handle its current load, why not open it up for anyone? So, after duly checking the university’s network usage rules, I slapped together a simple homepage for the server itself, advertising free web and FTP space for all who ask, and Dragonfire came into being. (To be precise, at this time it was still called Dragon, from the hostname dragon.res.cmu.edu; the name Dragonfire would not appear for a few months yet.)
The following month of November was probably one of the quietest of Dragonfire’s life. Account requests slowly started trickling in, perhaps one or two a week at first. With each one, I’d read over the account request, add the user account, create the necessary directories on the server, update various configuration files, and add the new user’s information to the user database—nothing more than a short text file. Overall, it was about a 5-minute process, as I recall: certainly nothing to worry about at the rate requests came in, and if anything, the excitement of someone else finding the site and expressing interest in using it far outweighed the loss of time.
That peaceful month was abruptly ended by a warning I received from the university’s data communications department, or DataComm, as they were known: “Pirated software has been found on your server. Remove it now, or else.” This was, of course, quite a shock to me, since I certainly had no intention of engaging in such unscrupulous behavior. I don’t recall whether the offending URL was pointed out to me directly, or whether I found it by perusing the server’s log files; but either way, one recently-added account’s FTP traffic stuck out like a sore thumb, dwarfing the others over the previous several days. This discovery was a big blow to my youthful innocence, but having become aware of the problem, there was nothing for it but to delete the account.
Just deleting the account didn’t really strike me as fair, however, no matter how improper the owner’s behavior was. So I proceeded to write up a simple set of rules delimiting what would and would not be permitted on Dragonfire, and what the penalties would be for violating the rules. Even at the time, though, I was aware of—in fact, had had to read myself—rule lists and policies full of so-called legalese, which was, if not quite impenetrable, certainly no fun to read. So I drew a line there, establishing a basic tenet of Dragonfire administration which would hold at least as long as I was in charge: mutual trust. I don’t recall whether I explicitly said as much in that first set of rules, or whether that came later, but I took the position that the rules were written with common sense in mind. We (the royal We, though at the time I was the only one on the provider side) asked Dragon’s users to trust us to apply the rules fairly, and in return we trusted users to interpret the rules fairly. I can’t imagine a system like that would even get off the ground today, but at the time, I wanted to make a statement: I wanted to show that you didn’t need legalese for everything—or die trying, so to speak.
Speaking of traffic, one thing I had been doing ever since I set up my FTP server back in high school was keep track of the server’s data transfer statistics. Traffic analysis is, of course, an excellent tool for detecting problems, as the pirated software incident amply demonstrated; but to me, it was also an indication of the server’s popularity—and therefore of how useful others were finding my efforts. And to be honest, I have always enjoyed playing with numbers, be they traffic statistics, finances, or abstract mathematical problems; so it was only natural that I would write a short program to summarize transfer statistics from the FTP and web logfiles.
And what a surprise those statistics provided. While transfers had been gradually growing even during the first months of modem-based connectivity, the growth turned virtually exponential after my move to a high-speed link. I had known firsthand about the traffic increase from the Squaresoft site, of course; traffic that month jumped by a factor of five from the previous one, to a level that would have saturated the modem connection (and undoubtedly angered the school’s staff). But even after that initial jump, traffic continued to roughly double each month over the next few months. During its first year of operation, Dragon grew from a small FTP site transferring 100MB in a month to a high-speed server sending out 1GB each day. These days, multimedia files reach into the gigabyte range easily, and with a sufficiently fast connection downloading such a file is a matter of mere minutes5; but for a freshman college student in the mid-1990s, it was difficult to conceive of a whole gigabyte of data going somewhere every day, especially from a single little machine sitting under the desk in my dorm room.
As the year drew to a close, word about Dragon seemed to be spreading. The rate of incoming account requests grew, slowly but surely; by the end of December, I saw one or two requests a day, and the excitement of having another Internet user request an account slowly gave way to the tedium of having yet another Internet user request an account. Naturally, the programmer’s weapon of choice against tedium is automation, and around this time I wrote the first scripts to automate account management. The user database was still a flat text file, but instead of adding new entries manually, I taught myself the basics of the Perl scripting language and wrote a script to take care of the details for me. (This script would give me some nasty headaches later on.)
All in all, it was a moderately eventful year, but an interesting one nonetheless, and things were looking up. My first semester of college ended in mid-December, and I headed home for the holidays, trusting that my Linux server would manage to keep itself running for the month I was away.
As it turned out, the server itself performed just fine on its own. My dormitory room Ethernet connection, however, did not fare as well, providing me with a bit of a scare in early January when Dragon suddenly became unreachable. I initially feared that the server had gone down, which would be a significant problem in terms of service, since I would not be able to return to the dorm to fix it until the middle of the month. A call to DataComm, however, revealed the true cause: My connection had been administratively disabled in response to a report of a cracking attempt from my IP address. The report, it turned out, came from the owner of a system named dragon.net.
As my server’s traffic increased, I had started considering the possibility of getting my own domain name. The logical choice, of course, would have been dragon.net, but as I had quickly discovered, that name was already registered. Curious as to what the system was, I tried connecting on some common service ports: HTTP, FTP, SMTP, NNTP, Telnet, and finger (the latter three were still in fairly common use at the time). Except for SMTP, all of the connections were refused, so I shrugged it off, figuring that it was a private system. Apparently, though, the system’s owner took that as a break-in attempt and reported it to Carnegie Mellon, who subsequently disabled my connection.
The DataComm staff member I spoke with explained the situation to me, patiently telling me that my actions were akin to knocking on locked doors and windows of a house, and I must behave more circumspectly on the Internet. I could naturally understand how a system owner might be paranoid about break-in attempts; given that students were on vacation, I certainly can’t fault DataComm’s response to the complaint itself; and indeed, I blamed myself for not having foreseen that my actions could have been perceived as hostile. At the time, I was more concerned with getting my server back online than with defending my actions, so I gave the staff member a meek “yessir”. Nonetheless, I would still respectfully disagree with his characterization: To the extent that a computer—especially one associated with a domain name—is accessible to others on the Internet, the system administrator should expect people to attempt to access it on well-known ports. Many physical metaphors have been applied to the Internet with varying success, but if one were to use the brick and mortar analogy, I would argue that my actions were closer to checking a storefront on a shopping strip for an “Open” or “Closed” sign, hardly an unexpected or suspicious event.
At any rate, the connection was reactivated, and Dragon once more became visible to the world. I settled back to enjoy the last week of my winter vacation—nearly lengthened due to surprise snowstorms that blanketed the area—and returned to Carnegie Mellon in mid-January to begin my second semester of college.
Account requests continued to come in unabated, and along with them came support requests: questions from users on how to make use of their account, requests to change account information, and so on. The latter, in particular, came to consume a not-insignificant chunk of time from my day; so, in order to free myself for other pursuits (such as schoolwork), I enlisted the help of a friend to take care of handling those requests. I may have been a bit nervous about delegating responsibility for the first time, but Dragon was still a small server with only around 100 users, so I didn’t expect much trouble. Based on mail from the period, things do indeed seem to have gone smoothly, freeing up time for me to deal with issues like all-too-frequent memory shortages.
As the number of users grew, I also became increasingly concerned about the potential for data loss. I was familiar enough with the concept of backups, but the idea of copying more than 1GB of user data onto stacks of 1.44MB diskettes6 didn’t really appeal to me. Instead, I managed to secure a tape drive for which I was able to buy tapes at the campus computer store; I can’t recall any longer exactly what kind of tapes they were, but they held 500 megabytes of data each, significantly reducing the amount of time I had to spend making backups.
At the same time, I was still looking into getting a domain name for my server. I explained the situation to some friends on IRC one night, after learning why the server had disappeared, and one of them suggested dragonfire.net as an alternative to dragon.net. The name rather appealed to me, and—more importantly—it wasn’t in use at the time. Domain speculation7 was unheard of in the 1990s, so acquiring an unused domain was simply a matter of establishing nameservers and sending the appropriate form and payment to InterNIC, the DNS management authority.
Filling out forms was simple enough, but registration of a domain required two functioning nameservers, and I only had one machine. Another friend from IRC, Ian Justman, came to the rescue, offering to set up a “West Annex” (so named since he lived on the west coast of the USA, while I was located on the east coast) for Dragon on his own home network. This allowed me to establish the second nameserver I needed, and also gave me a backup site from which I could at least post status messages if my own server became inaccessible again. Though I had no way to guess at the time, this second ability would prove critical later in the year.
With everything prepared at last, I submitted the domain registration request, and on February 26, 1996, Dragon transformed into Dragonfire Internet Services.8
Aside from the name change and the addition of a logo (shown at the top of this retrospective) to the server’s homepage, nothing particularly momentous occurred to mark the occasion. User accounts continued to increase at a steady pace, however, and the Linux box which had served me so well until then began to show signs of stress.
As a result of earlier system resource problems, I had already created a simple system status display program, with output looking something like this:9
|1:46am up 14 days, 1:21, 4 users, load: 0.03, 0.05, 0.08, 148 procs Memory use: 320200/2074416 real + 903484 buffers 0/32760 swap Disks: / 81 5329468/ 6613856 139844/417792 33 /home 77 10147576/ 13228720 68829/829056 8 /data 65 52361632/ 79965468 11682/234624 5 /scratch 21 65862408/310681652 68530/ 75872 90|
(The numbers above are from my personal machine in late 2007; a system with specifications like these would have been inconceivable in 1996!) By this time, both 1GB disks installed in the server were hovering consistently in the red “danger” zone. So I managed to secure a 2GB hard disk to add to the server, which was delivered in mid-March; after a considerable amount of physical and logical rearrangement, the server once more had room for growth, at least in the disk space department.
Also around this time, I made an addition to Dragonfire’s rules that is without doubt one of my more questionable decisions. This addition was the “good citizen” clause, which threatened Dragonfire users with account termination for “inappropriate or illegal” activities—whether performed using a Dragonfire account or not. I don’t recall exactly what prompted me to add this clause, though mail from the period suggests that there were some users trying to play fast and loose with the rules and I got tired of their games. Either way, the rule essentially served as an excuse for me to remove users I didn’t want on Dragonfire. It goes without saying that no proper commercial provider would even consider a clause like this; I may have felt that since I was providing accounts for free, I had the right to be more pushy about such issues. (Even if so, I retained the rule after I began charging for service, so that hardly qualifies as an excuse.) Perhaps the only saving grace is that, aside from whatever incident may have prompted its creation in the first place, I don’t recall any occasions on which I actually made use of the rule—though that hardly mitigates the chilling effect it must have had on users. In any case, among the many things I would change if given the opportunity to redo Dragonfire, this is close to the top of the list.
At any rate, the approach of April meant that I needed to secure some sort of Internet connection for Dragonfire over the summer, since I couldn’t simply leave my system at the university for the whole three-month summer vacation. I had already investigated possibilities for connectivity, and settled on a 112-kilobit dual-channel ISDN10 connection as the best balance I could strike between service and cost; Dragonfire’s average bandwidth use was already close to 300kbps at the time, but ordering the requisite T1 line for something that provided no income whatsoever was far beyond my means. I made the necessary preparations, registered for an account with my local service provider, and ordered an ISDN line from the local telephone company, Bell Atlantic. The salesperson I spoke to assured me that installation of the line would be a matter of one to two weeks, and since I still had over a month before the semester ended, I figured that I was all set.
Come May, though, the installation was still pending; when I was finally able to secure an installation date, that date was set for May 22—eight full days after my departure from Carnegie Mellon. Frustrated, I sent a message to users explaining the situation and apologizing for the interruption in service. Most users were understanding, though, which was at least a slight relief to my jangled nerves. I took advantage of the West Annex system to post a status page informing visitors to Dragonfire’s site about the problem, and settled back to await the installation.
One day before the planned installation, though, I received an unexpected call from the network provider which was working with Bell Atlantic on the installation. According to the call, Bell Atlantic’s engineers had, that very morning—over a month and a half since I had initially ordered the line—discovered that the local telephone switch did not support ISDN and would have to be upgraded before the installation could proceed. As a result, the installation had been delayed to “early the following week”. My frustration mounted as I was once again forced to type out an apologetic explanation to Dragonfire’s users. And predictably enough, the complaints started trickling in: “Please remove my Dragonfire account.” “Will Dragonfire ever be back up?”
Another week passed, and there was still no word about installation, so I called up the network provider again. This time I was told that Bell Atlantic had “just discovered” that my house was too far from the telephone switch for a straight ISDN line, and would have to install a repeater to amplify the signal. Naturally, this meant yet more delay. In the end, the ISDN line was not connected until June 13, thirty days after I had brought the system home.
In perhaps cynical retrospect, I have no proof that Bell Atlantic was actually the party at fault; it’s also possible the network provider had simply forgotten and was shifting the blame to the telephone company instead of admitting their own faults, though I recall the person I spoke with sounding just as frustrated with the lack of progress. And in all fairness, ISDN was still a new technology at the time; my father (who took care of paying the ISDN bills until I had earned enough to pay him back) tells me that it took months for Bell Atlantic to get their billing straightened out. Either way, the end result is that Dragonfire suffered a full month of downtime before finally reappearing on the Internet.
Despite this—and to my considerable surprise—the vast majority of users (at least, the vast majority of those who responded to my status-report messages) were both understanding and supportive. I suspect that in any case, I would have continued running Dragonfire out of a sense of duty; by this time I had realized, at least subconsciously, that I was stuck with what I’d built—I couldn’t just take the system down while others were depending on it. But those messages of support, the friendly comments and encouragement sent by so many users, were an immeasurable boost to nerves that were frayed close to the breaking point. I can’t recall whether I ever expressed my thanks for that support, but if any Dragonfire users of the time happen to read this retrospective, allow me to say: Thank you. I really can’t express how much those messages meant to me.
Having ridden out that stormy month, Dragonfire once more got back to work. Due to various wiring issues, I had to set up Dragonfire in the basement of our house, but this didn’t particularly bother me as long as the server could do its job. In fact, I spent much of my time at home working at the console—proving that there’s at least a modicum of truth to the stereotype of computer geeks living in their parents’ basements:
Dragonfire and its owner in August 1996. The tower case is the server itself, with an external 4GB hard disk (low white box) attached on the right. The black box just barely visible on the shelf above the monitor is the ISDN router, an Ascend Pipeline 50.
In contrast to that first month of frustration, the remaining two months of summer vacation proved mostly uneventful. Naturally, the bandwidth was nowhere near enough to support all the sites hosted on Dragonfire; the ISDN line reached its saturation rate of 112kbps for a few hours on the same day it was connected, even though the updates to the dragonfire.net DNS data had not yet completely propagated. I was able to improve performance by modifying the HTTP server software to limit the bandwidth used for image files, giving preference to the HTML pages themselves; after a bit of fine-tuning, I was able to get a 100% response rate on HTML files, though at the cost of rejecting most requests for images. (At this time, many websites were still text-oriented, using graphics for decoration at most. Doing this sort of throttling on a modern website could easily render it unusable.) Nonetheless, new users continued to request accounts, and in August I had to buy an external 4-gigabyte hard disk to augment the 4GB of space already installed in the server.
August drew to a close, and at last I was able to bring Dragonfire back to the high-speed environment in which it had been born. As a typical college student, of course, I was not looking forward to the return of classes and homework, especially since I still had several required courses to clear before I could take the more interesting advanced lectures. As Dragonfire’s administrator, however, it was a relief to finally be free of that constricting ISDN connection, able to stretch my wings once more. (Well, not my wings, but you get the idea.)
The Internet community, it seemed, was just as relieved; no sooner had I reconnected Dragonfire than new account requests started literally flooding in—sometimes dozens in a single day. Just three days after returning to CMU, Dragonfire reached 1,000 user accounts, stretching that old user management script of mine to the breaking point.
As mentioned earlier, the “user database” on Dragonfire was nothing more than a simple text file listing each user’s username, account name11, and other details. My maintenance scripts worked by reading in the entire database, making whatever additions or changes were requested, and writing the file back out to disk. For one reason or another—perhaps I noticed an error and thought to cancel a change I was making—I interrupted the script as it was rewriting the database, with the predictably catastrophic result that most of the user database vanished into thin air. I was able to reconstruct the data from backups and directory listings, but the incident told me in no uncertain terms that I needed a better way to manage user information. So as classes started, I put all the time I wasn’t doing homework (and probably some time I should have been doing homework) into creating a more robust user database framework.
I’m not sure why I didn’t switch to a proper database management system. I had already gained some experience working with databases in my summer job; it may be that I felt such database systems to be too complex for a “simple” task like user management, or that I thought it would take too long to learn my way around them when I needed a solution now. It’s also quite possible that I just wanted to try coming up with something on my own, to experiment and learn for myself what worked and what didn’t. In any case, I ended up writing a full-fledged user data management library, complete with menu-based administration tools, to replace the old, crusty scripts I had been using.
The creation of a stable, well-defined interface to the user library gave me another idea: How about adding an automated account request form to Dragonfire’s homepage? I could easily write a CGI program to process the form data and inform the user if their requested username or account name was in use, avoiding the necessity of mailing back and forth to ask for alternate names. So I did.
If two to three dozen requests a day constituted a flood, the addition of that request form unleashed what I could only call a raging tsunami. The request rate quickly reached fifty per day, then a hundred. By mid-November, Dragonfire had swelled to over 5,000 user accounts—an increase of 4,000 in less than two months. When I finally announced a temporary hold on new accounts later that month, the rate was over 150 new accounts a day. I had streamlined the account creation process to the point where I only had to press two keys to confirm each account, and it still took a significant chunk of time out of my day.
The hold on new accounts was due only in part to this administrative workload. Another reason was hardware limitations; predictably, the rapid addition of so many accounts was a significant strain on the server. I replaced the outdated 486-based computer with a Pentium machine in November, relieving the worst of the immediate problems, but it was clear that even the Pentium had limits, and it wouldn’t last long at the rate Dragonfire was growing.
The final, critical reason for the hold on accounts was bandwidth use. At the time, each dormitory was connected to the main campus network with a single, shared 10-megabit connection. As I learned through several increasingly insistent complaints from CMU DataComm, Dragonfire was—incredibly—using over a third of that bandwidth all by itself. Due to technical limitations12, the actual throughput on the so-called “10Mbps” dormitory network was actually closer to 4Mbps—meaning that everyone else in my dorm had to compete for the tiny fraction of bandwidth Dragonfire wasn’t using. In the end, it came down to: Reduce your bandwidth use, or we’ll cut you off.
Naturally, the hold on new accounts would help prevent Dragonfire’s bandwidth use from growing, at least in the dramatic way it had during the previous months. But how could I reduce bandwidth use? As a stopgap measure, I throttled back the rate at which the server responded to HTTP requests; this was effective in the short run, but had the undesirable side effect of slowing down accesses to the server. As the semester drew to a close and I headed for home, I continued to worry13 over the bandwidth problem, trying to find any way around what I had begun to realize was the only viable solution.
That “only viable solution” was, of course, charging for service. Especially given the problems that had plagued Dragonfire over the past year, I knew that any attempt to charge would be greeted coldly. (I had conducted an informal poll over the summer regarding the then-hypothetical possibility of charging fees, and nine out of ten who replied had had no objection; but the number of replies was only a fraction of Dragonfire’s userbase at the time, which itself was insignificant compared to the 5,000 users who inhabited Dragonfire now.) As I headed back to start the spring semester of my second year of college, I thought over all the steps I would need to work through in order to make the transition.
I had already discussed with my network provider the possibility of colocating the Dragonfire server, and while I hadn’t signed the actual contract yet, I had the information I needed on that end: the price of connectivity, $500 per month for the bandwidth Dragonfire required. I added to that the rough cost of equipment addition and replacement, factored in taxes, and came up with a rough figure of $10,000 per year in total costs.14 (To a simple college student like myself, it was quite startling to realize that I would soon be dealing in four- and five-figure amounts on a regular basis.) So the critical question was how to balance the equation on the other side: How many users would be willing to pay—and what would I have to charge them?
A quick survey of the automated traffic statistics showed me that about half of the 5,000 users weren’t using their accounts at all; presumably they had found a site giving out free accounts and just signed up, figuring that having an extra account never hurt. That left 2,500 potential paying users, but certainly many of them would choose to move to another free provider instead of paying. Making a rough guess from my poll of the previous summer, I estimated that between 10% and 20% of those, or 250 to 500 users, would pay for service. If I assumed 400 paying users, a yearly fee of $25 would cleanly cover the expected costs; if the number of users fell to 250, I would probably be able to reduce my bandwidth charges and still stay out of the red. And if I was lucky enough to keep 500 users, I just might be able to earn back some of the money I had spent on Dragonfire in the last year and a half.
So I set the price for accounts at $25 per year, or $5 per two months—a slight premium in the latter case to cover the administrative effort of having to update the account data more often. Since many users had already donated money to Dragonfire, I decided it was only fair to count those amounts toward account fees; but at the same time I wanted to make as clean a break as possible with the current, free accounts. So I decided to ask all users, even those who had donated, to pay for at least two months of service; those who paid would then have any previous donations counted toward their account fees, extending the account’s expiration date. I also asked users to mail their payments, since credit card transactions over the web turned out not to be feasible. (A bonus to this was that many foreign users sent cash, and I got the chance to see numerous kinds of money from around the globe.) And I set the changeover date, on which all unpaid accounts would expire, to April 1, 1997.
With the details settled, I finally wrote out a message to Dragonfire’s users on January 20, explaining the situation. Steeling myself against the inevitable onslaught of angry replies, I sent the message out, and sat back to wait.
As expected, the response to this announcement was not nearly as generous as it had been during the previous summer’s downtime. Responses varied from the apologetic (“I’m sorry to go because I liked the services provided”) to the sarcastic (“If you seriously think I’m going to pay for such a slow server you better get a life.”) and the furious (which I just deleted after a cursory glance). Many users didn’t even reply at all, and most of the 5,000 free accounts expired when April arrived. But at the same time, some users were satisfied enough with the service that they sent in payments; and when the dust had settled, Dragonfire had about 380 paying accounts—coincidentally close to the “sweet spot” I had initially estimated.
On the network side of things, I had completed the paperwork for the connection with my network provider, and I scheduled the physical transfer of the system for late March, coinciding with the university’s week-long spring break. While the provider assured me they would have staff available 24 hours a day, I was quite nervous about moving the server so far away (not too far from my parents’ house, but about 400 kilometers from Carnegie Mellon). Until then, even if the machine acted up I could always log in on the console to fix the problem, or hit the reset button in the worst case. But with the server in a remote location, I had to rely on the machine’s network connection in order to do anything at all; if a freak software or hardware glitch cut the server off from the network, I was helpless. Nonetheless, the decision had been made, so I plodded along, trying to keep things moving as smoothly as possible over the transition.
By this point, I must admit that Dragonfire had lost a lot of its luster for me. As I said frankly to one longtime user who complained about my plan to charge for service: “Dragonfire has gone from something I thought of as a fun little hobby to a huge Service that’s absolutely no fun at all. And yet I can’t just shut it down, much as I want to, because so many people depend on it. I’m caught between a rock and a hard place: I can’t keep the server here, but at the same time I can’t keep it anywhere else because I’d have to pay huge amounts of money. So what else can I do?”
While my negative feelings about Dragonfire were undoubtedly exaggerated from reading all the complaints about fees, I can’t deny that the tedium of dealing with dozens of support requests every day, the frustration of having to deal with users that tried to skirt the rules, that sort of activity was wearing on me. At heart, I’m much more of a technical person than an administrative one, and the enjoyment of providing a service to others was rapidly being dulled. It was, once again, the kind, supportive messages from those users who stayed with Dragonfire that kept my spirits from collapsing completely. While most of the payment forms sent I received had only the requisite account details written on them, some users added friendly notes or sketches, which were always a joy to see. One user even sent a excellently drawn full-page illustration—in color, no less! (Color printers were still something of a luxury in those days.)
So I managed to weather those rough winds, and late in March, I disconnected Dragonfire from Carnegie Mellon’s network for the last time. I took the server home with me, as I had the previous summer; but instead of returning it to my parents’ basement, this time I took it to the site of Clark Internet Services, a.k.a. ClarkNet, my network provider from the previous summer and the company with which I had contracted for colocation.
Dragonfire’s initial site, as it happens, was on the floor of a barn. If I recall correctly, the founder of ClarkNet owned a farm, and took advantage of an unused barn to house the company’s network equipment. (When I first spoke with them and they talked about housing my server in “the barn”, I had blithely assumed that was their jargon for the room or building containing the equipment. I hadn’t imagined it was an actual, literal barn.) This only lasted for a week or so, however; power supply problems that coincidentally appeared around the same time led ClarkNet to move my server to a rack they were renting in another network provider’s data center, and I have to admit relief at having Dragonfire settled in a more traditional environment.
At last, April arrived. I allowed a two-week grace period for those who did not send payments early, so the transition from a free service was not instantaneous; but on April 17, over four thousand accounts disappeared from the server, and Dragonfire was a free provider no longer.
On one hand, going commercial meant that I was no longer constrained by the university’s network rules. In particular, I was able to lift the restriction against hosting commercial sites on Dragonfire, which the university would not have permitted. On the other hand, accepting money from users in exchange for providing service placed an additional onus of responsibility on me, one which frankly made me nervous. Saying “I’m doing the best I can” and expecting users to accept that is fine if you’re giving them something for free; but when money becomes involved, the service has to be good enough to satisfy those paying for it, and I wasn’t confident that I could manage.
As it turned out, Dragonfire’s first trial was an incident I had no power to resolve. Shortly after the changeover, the network backbone15 through which Dragonfire’s traffic passed suffered a DDoS16 attack, with the result that users trying to access pages on Dragonfire encountered slow response times. Nonetheless, I had to face numerous complaints of poor service from Dragonfire’s users. And who can blame them? Users shouldn’t need to care how service is provided to them, as long as it is. Frustrated at being unable to do anything, I poked and prodded both ClarkNet and the data center administrator, probably more often and more harshly than I should have; but in the end, the problem was resolved by the backbone company, and Dragonfire’s response speed returned to normal.
Once that issue was cleared up, things ran smoothly for the most part. There were a few occasional bumps along the way; in particular, ClarkNet continued having on-and-off problems with their power supply, to the point that the data center forced them to switch over to the common circuits. I also recall a couple of times I found Dragonfire down for no apparent reason, only to learn later that someone had accidentally knocked out a cable or switched off the wrong machine; though since ClarkNet’s rack was rather haphazardly set up (perhaps unavoidable when each customer has their own distinct machines and wiring), I guess I shouldn’t have been too surprised.
As time passed, though, the system load on Dragonfire began to creep up again, pushing against the limits of the hardware. It wasn’t the result of a flood of accounts this time; while new account requests were once more trickling in, the rate was much closer to that of Dragonfire’s first few months of operation. I can only guess that the increasing popularity of the World Wide Web as a whole had begun to make itself apparent.
One concept that had started making the rounds in web services was that of the personalized hostname. Until then, most providers (including Dragonfire) had given all their users URLs on the same server, with a distinct directory name for each user: for example, http://www.dragonfire.net/~achurch/. With personalized hostnames, each user would have their own hostname within the provider’s domain: for example, http://achurch.dragonfire.net/. This allowed users to simplify the URLs for their pages without having to go through the hassle and cost of registering a separate domain. Seeing as Dragonfire’s users were now paying for their service, I thought it only appropriate that I look into improving that service; and once I had modified the server software appropriately and obtained a block of 256 IP addresses17, Dragonfire began offering personalized hostnames.
Then, late in August, I was hit with a zinger: ClarkNet told me that they were changing their rates—and Dragonfire’s bandwidth would now cost not $500 per month, but $2,100.
This was quite a shock, not least because I had carefully tailored Dragonfire’s fees to the cost of colocation. If I kept to the same fee structure, the “not-quite-free” rate of $25/year would skyrocket to $100, no better than most other providers—and with considerably less stable service. Instead, I individually contacted the few users who were using the most bandwidth, explaining the situation and asking them to cover their fair share. With luck, I hoped, that would either help cover the added costs or encourage those users to reduce their bandwidth usage.
But by the time winter vacation rolled around, I knew that I wouldn’t be able to avoid passing the increased bandwidth charges on to users any longer. As my finance charts clearly told me, Dragonfire had been bleeding money in the months since ClarkNet’s rate change, and there was just no way to make things work otherwise. With a heavy heart, I set to working out just how much I would need to ask the users to pay.
In the end, I had to raise the basic account fee by about half, from $25 to $40 per year. At least I was able to keep it from jumping as drastically as my own costs had, but it was still frustrating to feel Dragonfire slip farther and farther from the inexpensive provider I had wanted it to be.
To try and take some of the sting out of the fee increase, I worked on several improvements to Dragonfire’s services. Most significant among these was the addition of a new server; the original server, Bahamut, had already been expanded and upgraded to its limits, and it was still nearly overloaded. On January 30—two days before implementing the new rates—I installed the new system at the data center, which finally took care of the system load problems for good. Continuing the fantasy theme, I named the new server Palantir, from J. R. R. Tolkien’s Middle-Earth novels.
With the addition of the new server and a growing number of service processes to monitor, I decided to merge my old system status display program with several other short utility scripts I had written, creating a full-fledged monitoring program18 which could show me the complete status of Dragonfire at a glance:
(Once again, the numbers19 are from my personal machines in late 2007, not the actual hardware used by Dragonfire.) Aside from having a compact status display, the program would beep at me if it discovered anything untoward, such as a server going down; on one occasion, that woke me in the middle of the night and sent me on a 12-hour trek from CMU down to the data center to repair one of the servers.
On the whole, 1998 was the most uneventful year in Dragonfire’s short history. However, I had one of my greatest surprises of all in May, when I received a letter from (what was at the time) a fairly major web service provider20 in the USA. I had almost tossed it out as advertising before I noticed that it was addressed to “President, Dragonfire Internet Services”. Upon opening and reading the letter, I discovered that this major provider was essentially offering to buy out Dragonfire.
I was amazed more than anything, both that Dragonfire had caught the eye of another provider and that the provider thought Dragonfire was worth enough to send such a letter. But at the same time, I couldn’t help but wonder whether they had done due diligence. Even after the February increase, Dragonfire’s rates were significantly below those of most other providers; taking on users and promptly hitting them with massive fees seemed to me to be asking for trouble.
Seeing no benefit to anyone, I decided to write back, pointing out why I didn’t think their proposal was such a good idea. If they were persistent, I figured that I could always see what the users of Dragonfire thought; perhaps the stability of a proven hosting provider was worth the additional cost after all. But I never received a reply, and the matter ended there.
There was one other kind of mail, albeit electronic, that I received from organizations occasionally: copyright and other claims against material posted by Dragonfire users, usually in the form of legal threats. I probably got about one every two months or so after taking Dragonfire commercial. At the time, the DMCA21 had not yet been enacted, so there was no standard procedure for dealing with problems like copyright violation; however, many ISPs of those days would simply delete any questioned material out of hand or even terminate the allegedly infringing user’s account, a heavy-handed practice—sometimes abused to “censor” undesired material—which I found repellent. As such, I took the position that, barring court orders or the like, I would take no action against any Dragonfire user unless I could personally confirm the alleged infringement (though I did pass along the allegations to the users involved).
While I can only look back and be amazed at my hubris, I replied to every one of those messages in a similar fashion: “Please provide proof of infringement, or Dragonfire cannot take any action.” Depending on the tone and content of the message, I sometimes even fell into lecturing mode, explaining my position. This must have been quite frustrating to the people at these companies who were just doing their job, undoubtedly thinking they were contacting a faceless administrator at a random provider who was also just doing his job. In fact, I recall one argument that went back and forth probably a dozen times as we each tried to convince the other of the “correctness” of our beliefs. Miraculously, though, I was never actually sued by anyone.
As my fourth and final year of college began in September 1998, Dragonfire was once again losing money. The fee increases I had implemented at the beginning of the year had not been enough to cover the higher costs of bandwidth—to put it simply, I’d goofed. Unfortunately, I wasn’t strong enough to admit that plainly, and I found myself unwittingly headed down the same road followed by many less scrupulous companies: using additional services as an excuse to raise base prices.
Unsurprisingly, most users weren’t fooled when I told users later in the month that “Dragonfire is pleased to announce” personalized hostnames becoming standard for all accounts and an increase in the default bandwidth limit—with an accompanying increase in fees. I did make a comment about “concerns regarding Internet connectivity”, but I suspect that may not have been convincing given the tone of the rest of the message.
At the same time, I had learned of and applied for an opportunity for an internship in Japan, eager for the opportunity to learn more about the country and try out my Japanese skills in a real-life setting. Of course, if I was accepted, I would end up being located halfway around the world from Dragonfire; this would be something of an impediment to Dragonfire administration, to put it mildly. I could, at least in theory, relocate Dragonfire to a colocation center in Japan and continue it there. But I began to consider the possibility that I might have to borrow others’ help to run Dragonfire—or perhaps let Dragonfire go completely.
So I began talking with a friend of mine, Mikel Tidwell, who also ran a small service provider, called Dreamhaven. We worked though various possibilities, such as having him help administratively while I was overseas (I didn’t plan at the time to remain in Japan indefinitely even if the internship came through), or forming a partnership to run Dragonfire. We didn’t finalize anything yet, though; I was still waiting to hear back regarding Japan, and more than anything, I was hesitant to turn over the reins of Dragonfire to anyone else.
In November, I received word that I had been accepted to the internship program. That set a firm time limit on when I would have to decide what to do about Dragonfire: I would be departing for Japan the following May, shortly after graduation. I began looking into what would be needed to set up an LLC, a simplified type of corporation, for Dragonfire; but I had not yet decided what I was going to do with the servers themselves.
That latter decision was made for me when Verio, an Internet company which had purchased ClarkNet, demanded administrative access to my servers. Since the buyout a year earlier, I had had a number of problems with Verio, and I simply didn’t trust them enough to give them full access to Dragonfire’s systems. So I talked with Mikel, and we decided to move the servers to the same place Dreamhaven was located, a provider in Utah. After finishing up the semester, I headed home and retrieved the servers from their colocation site; and the last I ever saw of Bahamut and Palantir was when I packed them up on December 28 for shipping to Utah.
I can’t recall exactly what gave me the final push toward my decision to let Dragonfire go. As I mentioned above, the administrative side of Dragonfire was a continuing burden on both my time and my spirits; I certainly wanted to go to Japan with as few extraneous things on my mind as possible. And the fact that the servers were now out of my reach may have made it easier to come to that final determination.
As I prepared to return to Carnegie Mellon for my final semester of school, I broached the issue to Mikel: Would you be interested in taking over Dragonfire?
He was delighted at the opportunity, and over the course of several more discussions, we finalized an agreement whereby I would turn Dragonfire over to him, and he would pay me a percentage of profits to cover the cost of the equipment. And on February 13, I officially left my post as administrator.
That choice was probably the best one that could have been made from the viewpoint of Dragonfire and its users. Mikel was technically competent, and moreover, he didn’t shy away from administration as I tended to do. He also had experience running his own (albeit small) provider, and he had worked for a larger, commercial operation as well. He had the servers under his direct control, so he would be able to resolve any problems far more easily than I could if I had to administer them remotely.
And yet, it was a hard decision to make. After all, Dragonfire (and Dragon before it) had been with me for over four years; for all the trouble it had been at times, I had grown rather attached to it. It would certainly be a relief not to have to deal with support request after support request, not to have to worry about whether one of the machines was a bit too loaded or one of the hard disks was getting a bit too full. But I would have no more opportunities to work on optimizing the server processes, no more chances to come up with another useful tool for users. I would no longer be able to glance at the system monitor screen and see everything running along smoothly. I didn’t have the faintest idea what I was going to do with all the time I’d normally spend working on Dragonfire.
In the end, though, what mattered was Dragonfire’s users—and if Dragonfire’s users were best served by my stepping out of the picture, then so it must be. I stayed around to help Mikel with the transition, of course; but Dragonfire had concluded its final chapter under my care, and I could only gaze after it as it flew off to new skies.
Looking back on Dragonfire now, I’m fairly incredulous that people were willing to publish their websites on what was frankly a shoestring operation, especially after I started charging for service and still couldn’t improve things much. Even for the time, Dragonfire probably ranked among one of the least stable service providers on the Internet.
While I honestly tried to provide the best service I could, I may well have hurt Dragonfire by setting my sights too low. As can be seen from Dragonfire’s website (see the FAQ on downtime), I explicitly disclaimed any intent to try for 100% uptime, even going so far as to bring up the adage “you get what you pay for”—with Dragonfire itself as the victim! I was only intending to be realistic, but such statements probably turned into a subconscious excuse to let problems slide by. In truth, I tended to wait until issues like lack of disk space or CPU power actually manifested themselves before taking action, rather than thinking ahead and preventing the problems from occurring in the first place.
I think that attitude may have stemmed from a hesitation to go—one might even say fear of going—truly commercial, in the broadest sense. I never really got over thinking of Dragonfire as a hobby, an experiment with which to occupy my free time, and thus I never seriously considered the possibility of expanding it as a business.22 I eschewed at least one opportunity to visit a trade show and learn from others in the industry—undoubtedly there were many such conferences I could have attended, but I still have the unused tickets and the invitation letter from ClarkNet to the May 1997 ITEC Expo. It’s been too many years since then to recall my thoughts of the time with any clarity, but I suspect that at heart, I didn’t really want to run a service provider for any extended length of time. The technical aspects were interesting, certainly, and I enjoyed working through the various difficulties I ran into (however much distress they may have caused the users); but as I commented above, I found the administrative aspects rather tiresome to deal with. My growing interest in Japan at the time was undoubtedly another factor; even before my internship was finalized, I was probably reluctant to tie myself to a job in the USA without investigating possibilities abroad.
In the decade since my experiences with Dragonfire, the World Wide Web—and the Internet as a whole—have changed dramatically. Websites are no longer just a cute little toy for people to play around with; these days, everything is on the web. (In fact, there’s so much on the web that it would be practically impossible to navigate without search engines like Google.)
Perhaps from the length of my experience with the web, I tend to be skeptical of the new fads that arise in web technology from time to time. From Netscape’s loved-then-hated <blink> tag to framesets and scrolling status bar text, any number of ideas have popped up and quickly vanished. On the whole, however, I’ve seen the web transform from a mostly static repository of information to a dynamic community of users. “Wikis” (such as the Wikipedia project), allowing users to collaborate in creating a website, are probably the most visible example23 of this, and are a striking demonstration of the Internet’s power to connect people.
On the hosting side, things seem to have pretty much settled down. There are still free hosting services, generally supported by advertising; I’m mildly opposed to the concept of advertising as applied to the web (though I don’t find the text-based ads common today quite as objectionable), which is one reason I didn’t seriously consider it for Dragonfire. The pay services seem to be generally run by larger companies, with little room for small players anymore—Dragonfire itself was squeezed out of the provider business late in 2000. On the other hand, the very concept of “web service provider” has grown and changed since the 1990s. There are still simple hosting services like Dragonfire was, but there are also application service providers, web site designers, and more. Now that the base Internet technologies are nearly ubiquitous, I suspect entrepreneurs will need to find clever ways of making use of those technologies in order to succeed.
So what’s in store next for the World Wide Web? That I don’t know. I doubt the often-hyped “semantic web” will materialize in the near future, at least in its full-fledged form; the natural language processing technology needed to make this useful is, like most advancements in artificial intelligence, at least 20 years off.24 With due respect to Tim Berners-Lee, I’m also not sure a semantic web is really needed, though I’m certain the associated research will prove useful for other ends.
Without a doubt, the web’s primary power lies in connecting people to information. I still remember an occasion, sometime in the early 2000s, when I became curious about a song I heard playing on a cafeteria loudspeaker; without thinking, I pulled out my mobile phone, brought up Google, and searched for a phrase from the song. Thirty seconds later, I had learned the song’s title and artist—and only then did I realize what I had just done: located exactly the information I was looking for out of an immeasurable flood of bits and bytes in almost no time at all. Before the advent of the web, I would have had to chance across someone else who knew the song and hope that they recognized it from a single phrase, a task so daunting I probably would have just given up.
More important than raw information, though, are the people behind it. Wikis aren’t just an information depot; they bring together the people behind that information, letting them find and work with others who share their interests. More recently, social networking services such as MySpace and Facebook have brought this aspect of the web to the forefront. As the web becomes more dynamic in nature, I expect to see increased emphasis on this sort of interpersonal connectivity.
I do have to admit to uncertainty over whether we, the human race, are ready for such a level of connectedness. Just a hundred years ago, instantaneous communication around the world was the sole province of a few telegraph operators, and countries (especially on different continents) were separated by so much travel time they might as well have been different worlds. Even now, despite the vast leaps made in technology, needless conflicts continue to arise out of simple misunderstandings or failures to communicate; and I fear that dropping such a powerful tool as the Internet into the middle of that mess will only exacerbate the problem—after all, a tool that can spread information can also spread misinformation. But even if it does prove disruptive, I believe we will eventually learn how to use it well, to help eliminate the misunderstandings that separate us.25 If nothing else, the teenagers of the future will probably just brush us old fogies off to some remote corner of the ’Net and go on their merry, enlightened way.
How the Internet has grown! Just 25 years on from running Dragonfire and about half that since first writing this retrospective, much of the world now lives in an Internet-centric society. Government services are available online, and sometimes only online; ride-hailing apps have overtaken taxi call centers; most people are walking around with about two orders of magnitude more computing power in their pocket than Dragonfire ever had in its data center rack.
But thinking back over the intervening time, I can’t really say anything took me by surprise. Each individual step was reasonably apparent from the previous one, and while I certainly put effort into staying abreast of current research and practice, that was more out of personal interest than necessity. Miniaturization of components and improvments in touchscreen technology created the smartphone, arguably a better candidate for the term “personal computer” than the IBM PC which begat that term; replacement of traditional mobile phones by smartphones encouraged the development of online services based around each person having their own computer; and while MySpace has effectively been consigned to history, Facebook now connects a significant fraction of the global human population. (And I would be remiss not to point out that, 13 years into my 20-year prediction, natural language processing still hasn’t advanced enough for the “semantic web” to be a thing.)
On the less pleasant side of things, my prediction of the Internet being used to spread misinformation unfortunately appears to have been spot-on. Who, in any Internet-connected society, has not heard about so-called “fake news” by now? While the term itself is a sort of misinformation in that it is often used to label inconvenient facts which the speaker wishes their audience to ignore, actual “fake news” in the sense of lies presented as truth has rapidly become a major societal problem. Worse, the fabrications are not limited to text; advances in image and sound processing technology have made it possible to literally put words in people’s mouths, to create convincing simulations of people saying things they did not in fact say. (We didn’t have the technology to simulate the moon landing in 1969, to reference a persistent conspiracy theory, but we might well be able to do it in 2021.) I fear this plague of misinformation will get worse before it gets better.
One additional trend I find a bit disturbing is what I might call a rush to collective judgment over violations of societal norms. In one recent example as of this writing (April 2021), video streaming service Twitch declared that it would act against users of its service who engaged in hate speech or similar undesirable actions, even if those actions were not performed on Twitch itself. While I imagine that this is more a knee-jerk reaction to recent events than a deeply-considered change of approach, it nonetheless sounds eerily similar to the “good citizen” clause of Dragonfire’s rules which I discussed under the “1996” section, and I find it just as concerning as in Dragonfire’s case—perhaps more so, because viewer subscriptions are a nontrivial part of income for at least some of Twitch’s users. Given how quickly people seem to claim insult from even well-intentioned discussion of sensitive topics these days, and how difficult it often is for a single user to challenge a large company—or secure proper reparations even if they are successful—would such users not fear engaging in open discussion, however objectively reasonable that discussion might be? That is the sort of chilling effect for which I regretted my own decision to add Dragonfire’s “good citizen” clause, and I find it no less relevant here.
At the same time, I don’t dispute that proactive silencing of troublemaking users may be necessary for the good of society. Even from the days of the first “flash mobs”, it was clear that the ability to instantly transmit information to a wide audience raised risks which traditional societal systems are ill-equipped to address. One might make the superficially sensible argument that judicial instutitions such as courts are the proper forum in which to censure troublemakers, but when they can make their trouble and be done with it before law enforcement is even aware trouble has been made, that offers little solace to those who have been injured by the trouble. In the particular case of disseminating misinformation, even punishment of the disseminator will not stop that misinformation from spreading; lacking any way to, perhaps, “inoculate” ordinary users against misinformation, it seems the best we can manage is this sort of collective defense, despite the chilling effects it entails. (Sadly, I have no solution to offer for this conundrum. So much for age begetting wisdom . . .)
Perhaps in my frustration at seeing us stuck in this “misinformation” stage, I have developed a fair amount of antipathy toward social media services as they are currently implemented. What ought to be a tool allowing us to talk with other people is instead abused to talk at other people, without expectation of a proper conversation—and perhaps even in search of controversy, since angry interchanges draw viewers and provide the “likes” which offer such a seductive, if shallow, sense of self-worth. In particular, I feel that Twitter and its emphasis on short messages encourage what one might call “animal reactions” rather than reasoned discussion, and while I grant that Twitter has helped people as well, I admit that I would not be at all disappointed to wake up one morning and find that it had shut down.
I do hold out hope that, since my intuition has proven correct in forecasting the current flood of misinformation, it may also prove correct in predicting that we will eventually learn to move beyond this stage, to truly understand the nature of this instant communication and how we can use it to build a better society for all rather than simply destroy that with which we disagree. I often find myself thinking back on a remark I wrote as a footnote toward the original end of this piece, on the topic of misunderstandings caused by instant communication, which I will quote here:
I don’t mean to suggest that we should eliminate the differences that lead to such misunderstandings; I, at least, find the view of a homogeneous human race espoused by some science fiction stories rather dystopian. It’s my hope that technologies like the Internet will help us better understand, and accept, each other’s differences. It is those differences, after all, that make human society so vibrant.
So many of the disputes I have seen in recent years feel to me as though they stem from a reflexive rejection of anything or anyone different from the speaker, to the point where I want to cry out, That’s not how it was supposed to be! Being both old enough to have lived during pre-Internet days and known the ideals of harmony to which that era aspired, but also young enough to have grown up around computers and have a native understanding of the related technologies and at least some of their ramifications, I have long envisioned a future in which Internet-style instant communication served as a ubiquitous adjunct to our lives, letting us bridge physical and social gaps even more easily and building an ever-wider, ever-more-diverse global society. And while I sometimes feel a touch of despair seeing people worldwide retreat into their own small, homogeneous tribes, I also realize that—much like pandemics, to reference another current source of concern—periods of trouble feel much longer to live through than to simply read about as past events. Indeed, it has been said that living through bad times is what reminds us to treasure the good times, and so I continue to hope that someday, we will be able to build that positive, diverse society. Even if I do get brushed off to some remote corner of the ’Net as one of those “old fogies”.
1“Baud” (also “baud rate”; named after Émile Baudot, an early telegraph engineer) refers to the number of symbols per second transmitted over a communication link. Early computer modems used a simple coding scheme with only two symbols, corresponding to the values 0 and 1, such that each transmitted symbol contained exactly one bit of information; in this scheme, the baud rate and data rate (number of bits per second) are equal. Later modems used more complex schemes which allowed more than one bit of information to be packed into a symbol; for example, modems communicating at 2400 bits per second typically ran at 600 baud, packing 4 bits of information into each symbol sent over the telephone line. Nonetheless, “baud” was misused by many (including this author, who didn’t know better at the time) to refer to data rate, perhaps because it was easier to say “2400 baud” than “2400 bits per second”.
2Serial Line Internet Protocol, a protocol used to establish Internet connections over modems and similar point-to-point links before PPP (Point-to-Point Protocol) became prevalent.
3Later known as Netscape Navigator. At the time of this writing (February 2008), Netscape is no longer being developed, though the currently popular Firefox browser was born from the Netscape source code several years ago.
4The source code to the final version of the server can be found in the Resources section of this retrospective.
5At present (February 2008), I routinely obtain sustained speeds exceeding 40Mbps in both directions through my consumer-grade fiber-optic Internet connection, and I can transfer 1GB files between my home network in Japan and a USA-based server in three to four minutes.
6Or “floppies”, as they are typically called these days (2008). Through the mid-1990s, at least, the term “diskette” was sometimes used to differentiate rigid 3½-inch disks from the flexible (“floppy”) 5¼-inch disks that had preceded them.
7The act of registering domains for the sole purpose of selling them to others at higher prices. At the time of this writing (February 2008), “dragonfire.net” is owned by a speculator asking US$320,000 for the domain. (December 2019 update: dragonfire.net became available for a much lower cost, and I have obtained it to use as a home for this retrospective.)
8With the name change, I also gained the opportunity to give my server its own hostname in the dragonfire.net domain. The name I chose, “Bahamut”, comes from a recurring dragon character in the Final Fantasy series of video games; given that many of the sites on Dragonfire at the time were related to video games or similar interests, it seemed quite appropriate.
9This program was based on an earlier script that simply ran the uptime, free, and df commands over and over. The first line shows the current time, system uptime, number of logins, load averages, and process count; the next line shows memory usage in 1024-byte units; and the disk status lines show percentage of data space used, used and total data space (in 1024-byte blocks), used and total number of inodes, and percentage of inodes used. Source code for the program is available in the Resources section.
10Integrated Services Digital Network, a technology allowing reliable, high-speed (for the time) connections over ordinary telephone wires by sending data digitally through the telephone network, skipping the analog/digital conversions necessary for modems which reduced data throughput. Potential customers of the time also expanded the acronym as “I Still Don’t kNow”, for reasons which will become apparent shortly.
11The “account name” was my way of working around the traditional 8-character limit on Unix usernames. I required each new user to select a username following the traditional Unix rules, but also allowed the user to select a different name—the account name—to be used in URLs; so, for example, a user could have the username johnpage but the URL http://www.dragonfire.net/~JohnsHomePage/.
12The network used simple Ethernet hubs to connect dormitory rooms, such that all computers in the building were effectively connected to the same physical communication link. This design introduces the risk of collisions: if two computers try to send data at the same time, the signals will interfere with each other and the data will be lost, forcing both computers to retransmit the lost data. As the amount of data transmitted on a link increases, so does the rate of collisions; eventually, collisions become so frequent that they consume most of the network’s capacity and the effective data rate begins to decrease. This enforces a “speed limit” on the network which can be significantly lower than the theoretical capacity.
13That worry was punctuated by one final untoward event: Dragonfire disappeared from the net shortly into winter vacation, only to reappear several days later. On the history page, I explained this as the apparent result of a malicious intrusion. I must, however, make an embarrassing admission: That may well have been a lie to cover a careless mistake of my own. Over a decade later, of course, there is no longer any way to know exactly what happened, but I have a distinct recollection of discovering that I had done something incredibly stupid, then frantically searching for any plausible excuse to explain it away. It’s also possible that I only thought I had made a mistake, and the system really was broken into; but even if so, I’m just as much at fault for not securing the system well enough. The details are, regrettably, too hazy at this point to say one way or another; but the fact that I have such a clear memory of something happening suggests that there’s more to the story than that single history file entry. If that’s the case, I can only say: Mea culpa. I screwed up, and I’m sorry.
14Those with business management experience may note one cost I neglected to consider: that of my own compensation for running Dragonfire. Whether because I was still a student and therefore didn’t consider myself worthy of a salary, because I simply didn’t need the additional income at the time, or for whatever reason, it never occurred to me to include a salary to myself in Dragonfire’s costs. By the time I had this omission pointed out to me, Dragonfire had already been running commercially for several months, and at that point I could hardly justify raising fees simply because I wanted to give myself some money.
15The primary “trunk” lines through which all Internet data passes. Much like the USA’s Interstate road network or Japan’s bullet trains (Shinkansen), the Internet backbones connect to each other and to the largest service providers; those service providers, in turn, connect to smaller providers like Dragonfire. Typically, data from a computer connected to the Internet will pass through four or five different networks before reaching its destination.
16Distributed Denial of Service, a type of attack that floods its target with so much data that the target becomes inaccessible. DDoS attacks are typically carried out via ordinary Internet users’ computers which have been infected with viruses or other malicious software, and (as in this case) can severely hurt even the largest networks—this is why it’s important for every Internet user to learn how to protect themselves from such malicious software.
17At that time, each separate hostname on a server was normally given its own IP address. A subsequent extension to the HTTP protocol allowed multiple hostnames to be served from the same IP address, and this latter extension is now almost universally used.
18Source code for the program is available in the Resources section. As a cursory examination of the source code will show, the only security on the control connection was a secret port number, a password sent in the clear, and source IP address checking. I’m lucky nobody figured out how to spoof TCP sequence numbers while I was using it.
19Astute readers will notice that two of the memory values (207441 and 104856) have the last digit truncated. At the time I wrote the program, PC-class systems with more than a gigabyte of memory were still far in the future.
20HostAmerica.com (no relation to the current owner of the domain). From what I can tell, HostAmerica was later bought by a provider called Sage Networks, which itself merged with another company named Interliant before going bankrupt in mid-2002.
21The Digital Millenium Copyright Act, a law passed later in 1998 which updated various parts of USA copyright law with respect to computers and the Internet. The portion of the DMCA relevant to this discussion is Title II, which grants service providers immunity from copyright liability if they promptly remove allegedly copyrighted material from users’ accounts when notified of its presence.
22In the end, Dragonfire never did earn enough money to recover my initial out-of-pocket expenses. While it brought in about $2,000 a year (net) over its commercial life, I was short $1,421 as of November 1998, the last month for which I still have data. Between the last three months I operated Dragonfire and the sale of Dragonfire itself, I may or may not have made up that difference, but either way, Dragonfire never turned a significant profit.
23I almost listed weblogs (“blogs”) instead, but it occurred to me that those are really no different in substance from the “bulletin boards” for which pre-Internet BBS systems were named. Everything old is new again, I suppose.
24One cynical view of this “20 years” phenomenon is that it’s just long enough for the person making the prediction to retire before being called on its inaccuracy. Having studied a bit of artificial intelligence myself, I think honest optimism is closer to the truth. Computer science in general is a field pervaded by optimism, which in many cases is justified; artificial intelligence is simply a branch of the field which has more roadblocks and unexpected twists than usual.
25I don’t mean to suggest that we should eliminate the differences that lead to such misunderstandings; I, at least, find the view of a homogeneous human race espoused by some science fiction stories rather dystopian. It’s my hope that technologies like the Internet will help us better understand, and accept, each other’s differences. It is those differences, after all, that make human society so vibrant.