1. Foreword and Introduction

1.1 Copyright

NoSQL RDBMS, Copyright (C) 1998 Carlo Strozzi. Part of NoSQL code comes from RDB, a similar package written by W.Hobbs, an has been included in NoSQL with permission from the author.

This program comes with ABSOLUTELY NO WARRANTY; for details refer to the GNU General Public License.

A copy of the GNU General Public License is included with this program, in the file COPYING.

1.2 Preface

This working draft describes, and provides instructions for the use of, NoSQL (I personally like to pronounce it noseequel), a derivative of the RDB DataBase system. The RDB system was (and still is) developed at RAND Organization by Walter W. Hobbs. Part of the NoSQL code, as well as large parts of the text of this document, have been taken directly from RDB, so a good share of the credit goes to the original author.

Other major contributors to the original RDB system, besides W. Hobbs, were:

Chuck Bush

Don Emerson

Judy Lender

Roy Gates Rae Starr

People who helped with turning RDB into NoSQL:

Vincenzo (Vicky) Belloli

David Frey

Giuseppe Paterṇ

Maurizio (Masar) Sartori

The NoSQL.jpeg logo has been kindly provided by Kyle Hart.

A special thank to Paul Lussier, who is contributing to NoSQL very actively.

While NoSQL Version 1 was very close to the original RDB system, both in the table format and in the program code, NoSQL Version 2 has departed radically from that.

The major differences between NoSQL V.2 and the old version 1 are:

• The table format has been simplified and is now upward compatible with the format used by other similar database packages, like Starbase and /rdb. The new format is largely compatible with RDB, except that comments are no longer supported in the table header, and the RDB column definition line has been replaced with a dashline. See section Data formats for more details.
• All the operators and utilities are now accessed through a common front-end command, nosql.

NoSQL tends to be biased in favour of Linux. This means that, wherever it matters, NoSQL makes use of the GNU versions of the various UNIX commands, as those are the ones normally found on this UNIX workalike. NoSQL is Free Software, released under the terms of the GNU General Public License. As such, it fully qualifies as Open Source Software.

1.3 Introduction

A good question one could ask is "With all the relational database management systems available today, why do we need another one ?". The main reasons are:

1. Several times I have found myself writing applications that needed to rely upon simple database management tasks. Most commercial database products are often too costly and too feature-packed to encourage casual use. There is also plenty of good freeware databases around, but they too tend to provide far more than I need most of the times, and they too lack the shell-level approach of NoSQL. Admittedly, having been written with interpretive languages (Shell, Perl, AWK), NoSQL is not the fastest DBMS of all, at least not always (a lot depends on the application).
2. NoSQL is easy to use by non-computer people. The concept is straight forward and logical. To select rows of data, the 'row' operator is used; to select columns of data, the 'column' operator is used.
3. The data is highly portable to and from other types of machines, like Macintoshes or MSDOS computers.
4. The system should run on any UNIX machine (that has the PERL and the AWK Programming Languages installed).
5. NoSQL essentially has no arbitrary limits, and can work where other products can't. For example there is no limit on data field size, the number of columns, or file size (the number of columns in a table may actually be limited to 32.768 by some implementations of the AWK interpreter, including mawk I think).

As its name implies, NoSQL is not an SQL database. The rationale behind this is well explained in the accompanying paper 4gl.ps (Postscript), or 4gl.txt (ASCII).

The data is contained in regular UNIX ASCII files, and so can be manipulated by regular UNIX utilities, e.g. ls, wc, mv, cp, cat, more, less, editors like 'vi', head, RCS, etc.

The form of each file of data is that of a relation, or table, with rows and columns of information.

To extract information, a file of data is fed to one or more "operators" via the UNIX Input/Output redirection mechanism.

There are also programs to generate, modify, and validate the data. A thorough discussion on why this type of relational database structure makes sense is found in the book, "UNIX Relational Database Management", Reference #2.

It is assumed that the reader has at least a minimum knowledge of the UNIX Operating System, including knowledge of Input/Outout redirection (e.g., STDIN, STDOUT, pipes).

This document presents information in the following order: The DATA section describes the structure of the data, with examples. There is a general discussion about operators in the section on OPERATORS, followed by several sub-sections, one for each operator in alphabetic order. Each has detailed instructions for use, and examples. There are sections describing selection of information using multiple operators and generating new tables.

Again, the key feature of NoSQL (and other similar packages mentioned in this manual), is its close integration with UNIX. Unlike other fourth generation systems, NoSQL is an extension of the UNIX environment, making available the full power of UNIX during application development and usage. This level of integration removes the need to learn yet another set of commands to use and administer the database system. A database is just a file, and can be maintained like all other files that the user owns or has access to. Because NoSQL commands are executable programs, the UNIX shell is inherited as the primary command language of the database; no other proprietary database scripting language, to my knowledge, is as powerful and flexible as the UNIX shell. The shell-level nature of NoSQL encourages casual use of the system, and successful casual use leads to familiarity and successful formal use. This concept is much more thoroughly explained in the paper "The UNIX Shell As a Fourth Generation Language", included in the NoSQL documentation tree with the file name 4gl.ps (Postscript) or 4gl.txt (ASCII), that shows why the UNIX shell is an excellent tool for scripting database access.

1.4 Perl and the Operator/Stream Paradigm

As stated in the Abstract, NoSQL uses the Operator/Stream DBMS Paradigm. The main reason why I decided to turn the original RDB system into NoSQL is that the former is entirely written in Perl. Perl is a good programming language for writing self-contained programs, but Perl's pre-compilation phase and long start-up time are worth paying only if once the program has loaded it can do everything in one go. This contrasts sharply with the Operator/Stream model, where operators are chained together in pipelines of two, three or more programs. The overhead associated with initializing Perl at every stage of the pipeline makes pipelining somewhat inefficient. A better way of manipulating structured ASCII files is using the AWK programming language, which is much smaller than Perl, is more specialized for this task and very fast at startup (on my Pentium II Linux /usr/bin/mawk (POSIX AWK) is just 99K. Perl 5 is almost 500K. You get the point).

1.5 Bug reports

Please send bug reports (fixes are most welcome :-) to nosql-bugs@mi.linux.it. Always include as much information as possible, especially the output of 'nosql version'. For 'bug reports' I mean not just errors in the code, but also grammatical mistakes, typos and bad English constructions in the documentation, as English isn't my native language :-)

There is a mailing list for discussions related to NoSQL. The address is noseequel@mi.linux.it. To subscribe simply send a message to noseequel-request@mi.linux.it with the word "subscribe" (without the quotes) in the message subject.