Special Edition
Using CGI

Written by Jeffry Dwight and Michael Erwin

with

Tobin Anthony, Danny Brands, Rod Clark, Mike Ellsworth, David Geller, Galen A. Grimes, Matthew D. Healy, Greg Knauss, Robert Niles, Bill Schongar, Crispen A. Scott, K. Mitchell Thompson, Matt Wright



CONTENTS

Introduction

Chapter 1   Introducing CGI

Chapter 2   Choosing Your CGI Tools

Chapter 3   Designing CGI Applications

Chapter 4   Understanding Basic CGI Elements

Chapter 5   Using HTML and CGI as a User Interface

Chapter 6   Examples of Simple CGI Scripts

Chapter 7   Custom Scripts for Forms Processing

Chapter 8   Modifying CGI Scripts

Chapter 9   How to Handle Custom Image Maps

Chapter 10   Using MIME with CGI

Chapter 11   Indexing a Web Site

Chapter 12   Using WAIS with CGI

Chapter 13   Custom Database Query Scripts

Chapter 14   Robots and Web Crawlers

Chapter 15   Generating HTML Documents in Real Time

Chapter 16   Using Server-Side Includes

Chapter 17   Person-to-Person Interaction

Chapter 18   Taking Advantage of Web-Based Audio

Chapter 19   Using Web-Based Video Cameras

Chapter 20   Tips and Techniques for Perl

Chapter 21   Tips and Techniques for Visual Basic

Chapter 22   Tips and Techniques for C and C++

Chapter 23   Tips and Techniques for Java

Chapter 24   Tips and Techniques for AppleScript

Chapter 25   Testing and Debugging CGI Scripts

Chapter 26   Error Handling in CGI

Chapter 27   Understanding CGI Security Issues

Chapter 28   Learning from the Pros

Appendix A   What's on the CD?

Appendix B   Commonly Used MIME Media Types

Credits



Special Edition Using CGI

Copyright© 1996 by Que® Corporation

All rights reserved. Printed in the United States of America. No part of this book may be used or reproduced in any form or by any means, or stored in a database or retrieval system, without prior written permission of the publisher except in the case of brief quotations embodied in critical articles and reviews. Making copies of any part of this book for any purpose other than your own personal use is a violation of United States copyright laws. For information, address Que Corporation, 201 W. 103rd Street, Indianapolis, IN 46290. You may reach Que's direct sales line by calling 1-800-428-5331.

ISBN: 0-7897-0740-3

HTML conversion by :
    M/s. LeafWriters (India) Pvt. Ltd.
    Website : http://leaf.stpn.soft.net
    e-mail : leafwriters@leaf.stpn.soft.net


PresidentRoland Elgey PublisherJoseph B. Wikert
Publishing ManagerJim Minatel Editorial Services DirectorElizabeth Keaffaber
Managing EditorSandy Doell Director of MarketingLynn E. Zingraf
Acquisitions EditorCheryl D. Willoughby Product DirectorsMark Cierzniak, Benjamin Milstead
Production EditorSusan Shaw Dunn EditorsKelli M. Brooks, Thomas Cirtin, Chuck Hutchinson, Anne Owen
Assistant Product Marketing Manager Kim MargoliusTechnical Editors Kyle Amon, Justin Bell Bill Crosbie, Eric Iverson, Geoff Mulligan, Brian Valente
Operations CoordinatorPatricia J. Brooks Editorial AssistantAndrea Duvall
Technical SpecialistNadeem Muhammed Software SpecialistOran Sands
Book DesignerRuth Harvey Cover DesignerDan Armstrong
Production TeamStephen Adams, Jason Carr, Anne Dickerson, Chad Dressler, Bryan Flores, DiMonique Ford, Trey Frank, Jason Hand, Sonja Hart, Damon Jordan, Daryl Kessler, Michelle Lee, Bobbi Satterfield, Kelly Warner, Todd Wente
IndexerCarol Sheehan


About the Authors

Jeffry Dwight is the CEO of Greyware Automation Products, a consulting firm specializing in custom applications and Internet-related utilities. He's a confirmed Windows NT bigot, and his firm produces NT software almost exclusively. Since he founded Greyware in 1990, the firm has become an important resource to the NT community. Jeffry is a certified engineer with expertise in dozens of operating systems and programming languages.

Jeffry also writes poetry and fiction, and is active in the science-fiction community. He chaired the Nebula Awards Novel Jury for the Science Fiction Writers of America (SFWA) in 1993 and 1994, and Greyware provides home pages for many SFWA authors-as well as SFWA itself. The Horror Writers Association, several genre magazines, and many other authors all make their homes at Greyware, too.

Jeffry is currently single, has no pets, and lives in Dallas. He enjoys programming and writing fiction, but would much rather give it all up in favor of mucking about with a guitar and a drink someplace cool, quiet, and dark.

Michael Erwin first worked with computers in the mid '70s. He built his first S-100 bus system in 1979, which combined his interests in electronics and circuitry design. It was based on the relatively new Intel Z-80, running assembler and later CP/M. In 1982, as a junior in Barboursville (W.Va.) High School, he helped develop several computer course curriculums for high school and adult students. In the fall of 1982, he began teaching adult classes at Cabell County Technical Center. During the spring of 1983, he was given the chance to install several local area networks in the local school systems as a pilot project.

He has helped start numerous regional computer-based companies and has designed various systems for the banking, chemical, environmental, manufacturing, and publishing industries. He has worked for Union Carbide, Pioneer Technology, and several branches of the U.S. government. In many of these organizations, he also developed the training and user education services.

A monthly columnist in Boardwatch Magazine, Mike has been a featured speaker at ONE BBSCON, where he has helped numerous others to become Internet service and Web space providers. Mike also currently works in the IT department of INCO Alloys International Inc. He's also a partner in eve, Inc., an ISP consulting firm, which has also given him the opportunity to publish "The WebMasters Resource" CD-ROM series. You can find Michael on the Web at http://www.eve.net/~mikee or you can e-mail him at mikee@eve.net.

Contributing Authors

Tobin Anthony holds a doctorate in aerospace engineering but has been tinkering with computers for more than 18 years, specializing in the UNIX and MacOS environments. A strict vegetarian, devout Roman Catholic, and lapsed private pilot, he spends what little spare time he has with his wife, Sharon, and three children, Michelle, Austin, and Evan. Tobin works as a spacecraft control systems engineer at NASA's Goddard Space Flight Center in Greenbelt, Maryland. E-mail and Web stops are welcome at tobin@pobox.com and http://pobox.com/~tobin.

Born and raised in the Amsterdam, The Netherlands, Danny Brands has a degree in chemical engineering and became a doctoral student at the University of Amsterdam's Department of Chemical Engineering in 1993. He set up the department's e-mail and Web server and started specializing in Windows CGI programming in Visual Basic. He has been active in several Web server related newsgroups and has done free-lance Windows CGI programming and Web development jobs in his spare time. When he leaves his computer, he is known to play a little on one of his Fender guitars, do some serious skating on one of the frozen Dutch canals, or drink Bulgarian cabernet sauvignon wine with his girlfriend, Ruth.

Rod Clark lives quietly in Seattle, where he does some contract programming and technical work, and maintains the Small Hours pages on the Web. He's worked as an electronic distribution specialist in the Seattle office of a company that has, so they say, occasionally misled people, and as a network implementation analyst during the startup of a local aircraft manufacturing plant.

Mike Ellsworth is the Development Manager of Advanced Technology and the Webmaster for the A.C. Nielsen Company. He established the corporate Web site and has developed two information delivery services for Nielsen: BrokerNet and SalesNet. While developing these Web services, he did extensive CGI programming, including interfacing with legacy systems. He holds a degree in psychology from Duke University and received writing training at the University of Denver. Mike and his family live in Minnesota, and he enjoys music, movies, basketball, running, and racquetball.

David Geller is the Director of Online Engineering at Starwave Corporation (http://www.starwave.com), the Internet's No. 1 content-based site hosting such popular services as ESPN SportsZone, NBA.COM, Mr. Showbiz, Family Planet, and Outside Online. David is also the author of several popular shareware programs, including SnapCAP (included with this book's CD-ROM), Origo, and WEB Wizard: The Duke of URL.

Galen Grimes lives in a quiet, heavily wooded section of Monroeville, Pennsylvania, a suburb of Pittsburgh, with his wife, Joanne, and an assortment of deer, raccoons, squirrels, opossums, and birds, which are all fed from their back door. Galen is the author of several Macmillan Computer Publishing books, including 10-Minute Guide to Netscape and 10-Minute Guide to the Internet with Windows 95, both published by Que Corporation; First Book of DR DOS 6, published by Sams; and 10-Minute Guide to NetWare, 10-Minute Guide to Lotus Improv, and Windows 3.1 HyperGuide, all published by Alpha Books. Galen has a master's in Information Science from the University of Pittsburgh, and by trade is a project manager and NetWare LAN administrator for a large international bank and financial institution. You can reach Galen by e-mail at gagrimes@city-net.com or through his home page at http://www.city-net.com/~gagrimes/galen1.html.

Matthew D. Healy does various tasks, from UNIX system administration and database administration to building Web front ends on top of Sybase, mSQL, and Illustra relational databases, at the Center for Medical Informatics, Yale School of Medicine. Before joining the center, he designed electric motor control systems, taught undergraduate biology lab sections, managed an AppleTalk LAN, and read many science-fiction novels. His other interests include skiing, folk music, filk music (science-fiction music that parodies folk songs), and the history of science. Along the way, he has earned a B.S. in engineering from Purdue University and a Ph.D. in zoology from Duke University. His e-mail address is Matthew.Healy@yale.edu; the URL http://paella.med.yale.edu/~healy/matt_healy.html is his home page.

Greg Knauss is a programmer working with Windows and UNIX. He lives in Los Angeles with his wife, Joanne. Previously, he contributed to Que's Using HTML and Special Edition Using Netscape 2.

Robert Niles is an independent consultant who's completely fixated on the Web, and on the Internet as a whole. Robert owns and runs a small business called ITM Services in Selah, Washington, that provides Internet-related services to those in the surrounding valley. Robert entered military service in 1984, completing an extensive one-year course at the Presidio of Monterey as a Czech linguist. After finishing military service, Robert returned home to Yakima Valley, where he lives with his wife, Kimberly; his son, Michael; and his daughter, Shaela. You can contact Robert at rniles@shaela.itm.com.

Overseeing online support for AimTech Corporation, Bill Schongar is always looking at new integration technologies to make life easier. When not on-line, he's off wondering how far a catapult can toss a head of lettuce, and other strange medieval thoughts to remove him from the modern world.

An independent hardware and software engineering consultant, Crispen A. Scott lists among his accomplishments such varied projects as the digital anti-skid braking system for the B-2 Stealth Bomber, various Windows drivers and applications, and embedded control systems for the medical and industrial control fields. Cris is currently developing Web home pages, CGI applications, and establishing Web sites for Chicago-based customers of his Commercial, Residential and Institutional Software Corporation. A graduate of the University of Tennessee, Cris also lectures, conducts seminars, and presents training reviews nationally. In his spare time, Cris continues to polish his writing skills in poetry and science fiction. Cris can currently be reached at crisin19@starnetinc.com and, in the near future, at his Web site (search for Chicago Developments).

K. Mitchell Thompson is an independent software developer in Atlanta with more than 12 years of commercial experience. In addition to this book, he has contributed to Using Turbo C++ 4.5 for Windows and Special Edition Using the Internet (first edition), also published by Que Corporation. Mitchell's current professional interests include Java, MPEG, and real-time and distributed systems. Any free time is devoted to his daughter Kate's piano, an Irish wolfhound's exercise, and conversation with his best friend and wife, Kathy. Mitchell can be reached at http://www.crl.com/~ktomsun.

Matt Wright works at Hewlett Packard, where he writes HTML and CGI programs. In his spare time, he maintains Matt's Script Archive, a Perl/CGI Web page located at http://www.worldwidemart.com/scripts/. Matt also enjoys skiing and fly-fishing with his parents and younger brother. He spent 12 years in Louisiana before moving to his current home in Fort Collins, Colorado.

Acknowledgments

From Jeffry Dwight: This book wouldn't have been possible without the help of Cheryl Willoughby, Ben Milstead, and the rest of the gang at Que Publishing, who lovingly and painstakingly pulled order from chaos and made everything come together correctly. That they did it at all is a miracle; that they do it regularly is beyond comprehension.

From Michael Erwin: First, I would like to thank my wife, Jacqueline, who has been a sweetheart when I have been working on this book. She truly loves me to be able to put up with my running around at Mach 2 with my hair on fire. Second, I would like to thank my mom and dad. I also thank Larry Brammer and Harold McKeand at eve Inc. for carrying my load during the last few weeks. I would also like to thank Gary Bratton and Suzi Bowen of Inco Alloys International Inc. for supporting me. Special thanks to Joyce Coombs, Mike Digman, and Todd Broucksou of MountainNet, Inc. And last but definitely not least, a big special thanks to Jack Rickard and David Hakala at Boardwatch Magazine for supporting me, and to also give me the opportunity to write for the best online magazine.

To everyone at Que: All I can say is wow. I especially would like to thank Cheryl Willoughby, Susan Dunn, Ben Milstead, and Mark Cierzniak for hanging in there and helping me along the way. Without them, you wouldn't be reading this now. Not to mention all the other contributing writers, who make this book the new standard for CGI programming.

We'd Like To Hear from You!

As part of our continuing effort to produce books of the highest possible quality, Que would like to hear your comments. To stay competitive, we really want you, as a computer book reader and user, to let us know what you like or dislike most about this book or other Que products.

You can mail comments, ideas, or suggestions for improving future editions to the address below, or send us a fax at (317) 581-4663. Our staff and authors are available for questions and comments through our Internet site, at http://www.mcp.com/que, and Macmillan Computer Publishing also has a forum on CompuServe (type GO QUEBOOKS at any prompt).

In addition to exploring our forum, please feel free to contact me personally to discuss your opinions of this book: I'm bmilstead@que.mcp.com on the Internet, and 102121,1324 on CompuServe.

Thanks in advance-your comments will help us to continue publishing the best books available on new computer technologies in today's market.

Benjamin Milstead
Product Director
Que Corporation
201 W. 103rd Street
Indianapolis, Indiana 46290
USA


Introduction

The Common Gateway Interface, or CGI, is one of the most useful tools in a Webmaster's kit. Whether you're the lone maintainer of a single home page on someone else's machine or the Webmaster of a huge domain, you'll find that CGI is essential for anything beyond presenting static text and graphics.

CGI is the magic behind Web-based interactive games, page counters, order-entry systems, online shopping carts, SQL database interfaces, animations, and clickable images. In fact, you'll find that CGI, in one of its many forms, is what brings the World Wide Web to life.

How This Book Came About

The book you are reading is the result of group effort: The lead authors, Jeffry Dwight and Michael Erwin; the production team at Que Publishing; the contributing authors; and you.

You? Yes-in two ways. First, the growing number of CGI programmers, mixed with proliferating standards and new Web server capabilities, told us there was both interest and need. The second way you contributed to this book is more specific: We based much of the book's content on the types of questions you most frequently ask in public places-feedback to program authors, Usenet newsgroups, chat sessions, CompuServe forums, and similar areas.

So if there hadn't been an interest from you-as well as a good deal of confusion and misinformation in the community at large-this book would never have seen the light of day. We hope to satisfy most of your questions and provide a reference you can keep around for project after project.

Intended Audience

We assume throughout the book that you have at least an intermediate understanding of programming in one or more languages. This book won't teach you how to program, but it will teach you how to use your existing programming skills to make CGI scripts work.

We didn't shy away from complicated topics, but we made sure to cover the fundamentals, too. In all, if you're comfortable with C, Perl, Visual Basic, or AppleScript, you should be able to glean a great deal of information from these pages.

We explain the basics of CGI programming fairly well, so even if you've never thought of writing CGI before, you'll do fine. If you're already an accomplished CGI programmer, you'll find hundreds of tips and tricks throughout the book to expand your repertoire.

Where possible, we used pseudocode or a textual description of the process under discussion. We did this for a couple of reasons: First, this book is intended to be platform-independent, meaning that you should be able to profit from it no matter what server you run and no matter what programming environment you use. Second, a textual description forces the reader (and the author) to focus on the process rather than the syntax. The goal is for you to understand how the magic works, not just what to type.

You'll also find a lot of actual code, both in the pages of the book and on the accompanying CD-ROM. We assume that programmers, once they understand the concepts, will want to go forth and create programs. What better way than to be armed with working samples?

How This Book Is Organized

This book is divided into eight major parts. These natural dividing lines let you hop around the book in hyperlink fashion, if that's how you like to read.

Throughout the book you'll find references to other chapters and sections where the material under discussion is either first introduced or discussed in more detail. This way, we can avoid recapitulating introductory material in an advanced section and can concentrate on the fundamentals without worrying about implementation in a tutorial section. In short, this scheme helps us keep from wasting your time. Feel free to jump straight in at the chapter that catches your interest, or read straight through from beginning to end. Either way, we have you covered.

Part I: CGI Fundamentals

The chapters in this section give you a complete introduction to CGI-everything from what it is and how it's used to what tools you'll need to start using it yourself. You'll also take a peek at where CGI is going, to help you prepare for the future.

Part II: CGI Application Development

The chapters in this section get right into the meat of programming CGI. We'll teach you how to design your programs, how to document them, and how to get them to work across platforms where possible. We'll also explain some of CGI's limitations, and teach you ways of getting around them.

Part III: CGI Programming Examples

These chapters start off by presenting some sample CGI scripts, and then show you how to modify them for your own use. You'll also learn about secure HTTP and how to take advantage of it.

Part IV: CGI Search Engines

The chapters in this section provide a basic overview of site indexing and database use, from both the user's point of view (finding information) and the Webmaster's point of view (providing information). You'll see why and how indexing is used, and find out all about tying your back-end SQL engine into your Web site.

Part V: Interactive HTML Documents

Interactivity is king on the Web, and the chapters in this section will show you how to make your site come alive with personalized, up-to-the-minute information. You'll also find a lot of ready-to-run sample programs that are both useful and fun.

Part VI: CGI Tips and Techniques

Here's where we let you in on the secrets of the experts. For each of the five most popular scripting languages, we show you the low-down, nitty-gritty details to let you get the most from the environment.

Part VII: Polishing CGI Scripts

The CGI operating environment places some special considerations on testing and debugging. In these chapters, we'll show you how to make your scripts as close to bulletproof as possible. We'll also talk about CGI security in detail.

Part VIII: Learning from the Pros

In this short section, we'll take you on a whirlwind tour of the Internet to show you places where you can see how the best and brightest minds have implemented CGI. We'll also point you to resources you can snatch and use for your own programs.

Conventions Used in This Book

Que has more than a decade of experience writing and developing the most successful computer books available. With that experience, we've learned what special features help readers the most. Look for these special features throughout the book to enhance your learning experience.

Several typeface and font conventions are used to help make reading the text easier:

This book also uses two icons to alert you to important information. The icon in the margin next to this paragraph tells you that the program, routine, library, or document under discussion can be found on the accompanying CD-ROM. We've included the things you'll find most useful and indexed the entire CD-ROM so that you can read it using your Web browser.

This icon alerts you to important security information in the text. You'll see it used to warn you of programmer mistakes that lead to insecure applications, common hacker methods and practices, and known security holes. Armed with these tips, you can make your programs secure and robust.

TIP
Tips suggest easier or alternative methods, to help you program more efficiently.

NOTE
Notes either point out information often overlooked in the documentation, or help you solve or avoid problems.


CAUTION
Cautions alert you to potentially negative consequences of an operation or action, especially if the latter could result in serious or even disastrous results, such as loss or corruption of data

Troubleshooting
What is the purpose of a troubleshooting section?
Troubleshooting sections provide you with advice on how to avoid or solve problems. Troubleshooting information is presented in the format of a question/problem followed by a solution.

If a paragraph mentions features described elsewhere in the book, a special cross-reference box appears next to the paragraph to refer you to a related section in another chapter in the book. These cross-references function like hypertext links and allow you to navigate through the text.

Sidebars Provide Deeper Insight
This paragraph format provides technical, ancillary, or non-essential information that you may find interesting or useful. Sidebars are like extended notes, but you can skip over them without missing something necessary to the topic at hand.

A Word from the Authors

Michael Erwin and I have enjoyed putting this book together. Our hope is that you'll find both instruction and inspiration in its pages and that, armed with the information we provide, you'll be able to write your own programs.

There are a thousand things we didn't cover, and if we had more space or more time, we'd love to include them. Our aim, though, was to provide all the essentials-to give you enough to get started. In the winnowing process, we had to make choices…this bit stays, that bit goes. We hope our choices make sense to you, and that you find this book both useful and fun. If you have suggestions for future editions, we'd like to hear from you.

-Jeffry Dwight, Dallas, 1996
-Michael Erwin, Barboursville, W.Va., 1996