Usage: nosql addcol newcolumn ... < table
The columns specified on the command line are appended to the right of the existing ones. The program does not check for duplicates.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql addcol Job < inputtable
Output on STDOUT:
NAME COUNT TYP AMT Job
---- ----- --- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql body < table
Strips the header off the input table, and prints the data (body) part to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql body < inputtable
Output on STDOUT:
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql cat [-d|--date datestring] table
Prints a NoSQL table the way it was at a point in time, by using RCS if necessary. The table must have been subject to versioning by RCS for this to work. Please refer to co(1) for details about the acceptable date/time formats. If the time string contains spaces, then it must be quoted, following the usual Unix shell quoting/escaping rules. If option '-d' is not specified, then the latest available version of the table is checked out. Examples of time specifications which are acceptable for RCS are:
A fairly common specification is "1998/05/21 LT".
This operator reads a table from a file and prints the requested version of the same table to STDOUT. The output stream can then easily be used to overwrite the original table on disk with the the following sequence of commands:
nosql lock table && nosql cat -d "1998/05/21 LT" table |
nosql write -s -o table && nosql unlock table
This will effectively roll-back the table to the desired point in time (i.e. the way it was before 1998/05/21 in the above example).
Note: locking and unlocking the table is not strictly necessary, but it should always be done in a multi-user environment.
Usage: nosql column columnname ... < table
Selects columns by name (and order) and outputs a table with these columns. Can effectively select, order, add, delete, or duplicate columns. If no columns are specified, then nothing is printed to STDOUT. If a column name does not match any of the columns in table, a new column with that name is inserted in that location. If the input table contains duplicated columns, with the same name but different values, only the first (leftmost) one will be printed to STDOUT.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql column TYP Job NAME < inputtable
Output on STDOUT:
TYP Job NAME
--- --- ----
A Bush
A Hansen
X Jones
B Perry
D Hart
D Holmes
Usage: nosql compute [-x|--debug] 'column = expression [; ...]' < table
Options:
Debug option. Print the actual AWK program to STDERR before executing it.
Computes values for data fields based on arbitrary AWK statements using column names. Any characters that are special to the UNIX shell must be quoted. Please refer to the documentation and man pages of your AWK implementation for more details on valid expressions. Comments, i.e. unquoted # signs in the AWK program are not supported.
This operator reads a table via STDIN and writes a table via STDOUT.
If the input table contains duplicated columns, with the same name but different values, only the first (leftmost) one is taken into account. The output will still have the duplicates, but this time with equal column values in them. If no expression is specified, then nothing is printed to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql compute 'COUNT = COUNT*10' < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 440 A 133
Hansen 440 A 23
Jones 770 X 77
Perry 770 B 244
Hart 770 D 1111
Holmes 650 D 1111
Note: in the AWK programming language, patterns in regular expression matching can be enclosed either in slashes (/) or in double quotes ("). With the 'compute' operator you sould always use the second form, i.e. you should not write :
nosql compute 'if(NAME ~ /^Hans.*/) COUNT=COUNT*10' < inputtable
as it may or may not work, depending on what is inside the /.../ block, but you rather have to use the quoted AWK form :
nosql compute 'if(NAME ~ "^Hans.*") COUNT=COUNT*10' < inputtable
The same considerations apply to other operators that parse the AWK program before calling the AWK interpreter, namely 'row', 'fcompute' and possibly others.
Usage: nosql dashline < table
Prints to STDOUT only the dashline of the input table.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql dashline < inputtable
Output on STDOUT:
---- ----- --- ---
Usage: nosql datatype [-x|--debug] < table
Displays the type and length of the data in each column of a table. Supported data types are:
String
Integer
Float
Currency
Date
Time
This NoSQL operator reads a table from STDIN and writes a table with the datatype information to STDOUT.
Example 1:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql datatype < inputtable
Output on STDOUT:
Field Width Type Description
----- ----- ---- -----------
NAME 6 S string
COUNT 5 I integer
TYP 3 S string
AMT 4 I integer
Example 2:
Input table inputtable
:
NAME TIME DATE NUM FLOAT USD
---- ---- ---- --- ----- ---
Bush 19:02:05 1998.09.13 12345 2345.234 1,234.01
Hobbs 19:22:05 98.09.23 -1234 12345.2345 0.56
Command:
nosql datatype < inputtable
Output on STDOUT:
Field Width Type Description
----- ----- ---- -----------
NAME 5 S string
TIME 8 T time
DATE 10 D date
NUM 5 I integer
FLOAT 10 F.4 float 4
USD 8 C currency
Usage: nosql edit [-K|--key field] [-u|--unique] table
This utility calls an editor to allow the editing of a table. The form of the file to be edited is the 'list' format, which is usually much more comfortable to edit than the 'table' format. The default editor is specified by the environment variable EDITOR if set, otherwise the editor 'vi' is used. EDITOR cannot be set to an editor that forks and detaches from the current shell (like xedit, 'vim -g', and the like). If you want to keep your current EDITOR value and still be able to use 'nosql edit' then you can set EDITOR to a non-forking editor in your ~/.nosql.conf file. See section Operators for details on setting NoSQL environment variables.
If option '-K field' is specified, when the table is written back to disk it is kept sorted on field. If '-K field' is not specified, then the table is kept sorted on the first (leftmost) field by default.
If option '-u' is specified, then besides sorting on the key field the table is also kept unique on the same field. Rows that are duplicated on that field will be silently dropped. This option should be used with care.
Care should be taken when editing, to abide by the 'list' format structure rules. See section Data formats for more details.
The table may be an existing file, or it may be automatically checked out from RCS. In the latter case it will be checked back into RCS after the editing is complete. The default action is that if the table does not exist an attempt will be made to find the table under RCS. To start RCS versioning on a table for the first time, the following command can be used:
ci [-u] table
From now on, edit
will handle all interactions with
RCS automatically.
Usage: nosql fcompute [-x|--debug] cfile < table
Options:
Debug option. Print the actual AWK program to STDERR before executing it.
This operator is functionally identical to the compute operator, but it reads the AWK statements from file cfile rather than from the command line. It can be used when the computing statements are too complex and it would be impractical to type them directly on the command line. The compute file may contain usual AWK comments, even inline ones. Regular expression matching in the AWK program file should follow the same rules described in the section about the compute operator.
Usage: nosql field from [to] < table
Takes a column number and outputs the corresponding column of the input table. If two numbers are specified, then outputs the corresponding range of columns. If the second number is lesser than the first, then all columns starting from the first column number to the last (rightmost) column are printed. If no columns or if a non-numeric or out-of-range column is specified for the first field, then nothing is printed to STDOUT. The same happens if a non-numeric column is specified for the second field.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql field 3 5 < inputtable
Output on STDOUT:
TYP AMT
--- ---
A 133
A 23
X 77
B 244
D 1111
D 1111
Usage: nosql fieldsof < table
This NoSQL operator reads a table from STDIN and writes the list of column names to STDOUT, one per line of output.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql fieldsof < inputtable
Output on STDOUT:
NAME
COUNT
TYP
AMT
Usage: nosql fromRDB [-doc|-o|--output file] < table
Options:
File name where to save RDB comments, header information and column documentation fields.
Converts an RDB table to NoSQL format. RDB header comments are NOT preserved during the conversion, as NoSQL tables do not support them. However, RDB table comments, column types and column documentation fields can optionally be saved in a separate file during the conversion process. The file format is compatible with NoSQL 'maketable' template files. Note that if a file with that name already exists it is simply overwritten without notice.
This operator reads an RDB table via STDIN and produces the NoSQL version of the same table on STDOUT.
Example:
Input RDB table inputtable
:
NAME COUNT TYP AMT
6S 5N 3S 4N
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql fromRDB < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql header < table
Strips the body off from a table and prints the header to STDOUT
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql header < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Usage: nosql headline < table
Prints to STDOUT the column name line of the table received via STDIN.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql headline < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
Usage: nosql help
This operator displays a message that tells how to access the NoSQL online documentation.
Usage:
nosql index table column ...
or
nosql index -u|--update [index_file ... ]
The first form of usage of this utility generates an index file for the column(s) given, that refers to the specified table. An index file is actually another (smaller) table containing only the column(s) given plus a column for index information. An index file can be used by the operator 'search' to quickly locate rows of data in the referenced table.
Index files are named by appending an 'x' and the column name(s) (separated by a dot) to the base name of the table it refers to. For example an index file for the table "area.rdb" on column "strip" would be "area.x.strip". An index file that was also on column "depth" would be "area.x.strip.depth".
The second form of usage of this utility updates the index file(s) given. If no files are given all index files in the current directory are updated. An update of an index file is necessary when the table it refers to has been modified.
This utility writes or rewrites tables with
defined names in the current directory.
NOTE: index
does not work on tables
smaller than three records, but it is not meant for them
anyway :-)
Usage: nosql inscol newcolumn ... < table
Inserts new empty columns in the leftmost position of a table. The columns specified on the command line are inserted to the left of the existing ones. The program does not check for duplicates.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql inscol Job < inputtable
Output on STDOUT:
Job NAME COUNT TYP AMT
--- ---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql islist [-v|--verbose] < file
Options:
Makes the error reporting more verbose.
Used by the 'edit' operator.
Checks that a file has a valid NoSQL 'list' format. This operator reads a file from STDIN and returns an exit code equal 0 if the file has a valid NoSQL 'list' structure, or 255 otherwise.
Example:
Input file inputfile
:
NAME Bush
COUNT 44
TYP A
AMT 133
NAME Hansen
COUNT 44
TYP A
AMT 23
NAME Jones
COUNT 77
TYP X
AMT 77
Command:
nosql islist -v < inputtable
Message printed:
list ok
and the code returned to the calling program will be 0.
Usage: nosql istable [-v|--verbose] [-n|--no-header] < table
Checks that a file is a valid NoSQL table. This NoSQL operator reads a file from STDIN and returns an exit code equal 0 if the file is a valid table, or 255 otherwise. If option '-n' is specified (together with option '-v'), then any failing row numbers printed to STDERR will not include the table header.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql istable -v < inputtable
Message printed:
table ok
and the code returned to the calling program will be 0.
Usage: nosql join [options] table1 table2
where table1
is the "primary" table, while table2
is
the "secondary" one.
Options:
Join on column from both files.
Join on column from file 1.
Join on column from file 2.
In addition to the normal output, produce a line for each unpairable line in file n, where n is 1 or 2 ("Master/Detail" join).
Exclude the join column (leftmost column) from the output table.
Debug. Print the join(1) program to STDERR before executing it.
Joins two tables on a common field, using the Unix join(1) program. The two tables must be sorted on the respective join fields for the operation to function correctly. Either one or the other, but not both, of the two input tables must be specified as '-', meaning STDIN. See join(1) for more details on the meaning of each option. If neither '-j' nor '-1|-2' are specified, then the two tables are joined on their respective first (leftmost) fields. If any of the two input tables contain duplicated columns, i.e. columns with the same name but possibly different values, only the first (leftmost) one is taken into account. The output will still have the duplicates, but this time with equal column values.
A natural join produces a new table that contains only rows from the input tables that match on the specified columns (key columns). A master-detail join produces a new table that contains all rows from the master table and those rows from the secondary table that match.
If different column names are specified, the name of the join column in the output table will be from table_1.
The order of columns in the output table will be: first the join column (unless '--suppress-join-column' is specified), then the other columns from table1, then the other columns from table2.
This NoSQL operator reads two tables, one from STDIN and the other one from a file, and writes a table to STDOUT.
If we have the table (named samplej) here:
name nr typ amt
---- -- --- ---
Bush 1 A 133
Bush 2 A 134
Hansen 3 A 143
Hobbs 4 B 144
Hobbs 5 B 144
Jones 6 C 155
Perry 7 D 244
Perry 8 D 311
and the table (named samplej2) here:
name cnt typ amt
---- --- --- ---
Hobbs 41 A 141
Hobbs 42 BB 142
Hobbs 51 BB 144
Hobbs 43 CC 143
then the command to do a natural join of samplej and samplej2 on column name is either:
nosql join -j name - samplej2 < samplej
or:
nosql join -j name samplej - < samplej2
and the result is shown in Table 6.
Table 6
Natural join of tables SAMPLEJ and SAMPLEJ2
name nr typ amt cnt typ amt
---- -- --- --- --- --- ---
Hobbs 4 B 144 41 A 141
Hobbs 4 B 144 42 BB 142
Hobbs 4 B 144 51 BB 144
Hobbs 4 B 144 43 CC 143
Hobbs 5 B 144 41 A 141
Hobbs 5 B 144 42 BB 142
Hobbs 5 B 144 51 BB 144
Hobbs 5 B 144 43 CC 143
The command to do a "masterdetail" join of the same two tables on column name is either:
nosql join -a 1 -j name - samplej2 < samplej
or:
nosql join -a 1 -j name samplej - < samplej2
and the result is shown in Table 7.
Table 7
Master-detail join of tables SAMPLEJ and SAMPLEJ2
name nr typ amt cnt typ amt
---- -- --- --- --- --- ---
Bush 1 A 133
Bush 2 A 134
Hansen 3 A 143
Hobbs 4 B 144 41 A 141
Hobbs 4 B 144 42 BB 142
Hobbs 4 B 144 51 BB 144
Hobbs 4 B 144 43 CC 143
Hobbs 5 B 144 41 A 141
Hobbs 5 B 144 42 BB 142
Hobbs 5 B 144 51 BB 144
Hobbs 5 B 144 43 CC 143
Jones 6 C 155
Perry 7 D 244
Perry 8 D 311
Note that these two forms are equivalent:
nosql join -j name ....
nosql join -1 name -2 name ....
Refer to the man page join(1) for more information on how the Unix join utility works. See also section pull for a faster alternative to 'join', that can be used in some cases.
Usage: nosql justify [-w|--width width] < table
Options:
Truncate output columns to width characters.
Makes a table more readable by adding spaces around fields.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT, left-justifying string columns and right-justifying numeric ones. Justification is performed by padding with spaces where needed. The output stream is still a valid NoSQL table, just with more blanks.
Example:
Input table inputtable
:
NAME TIME DATE NUM FLOAT USD
---- ---- ---- --- ----- ---
Bush 19:02:05 1998.09.13 12345 2345.234 1,234.01
Hobbs 19:22:05 98.09.23 -1234 12345.2345 0.56
Command:
nosql justify < inputtable
Output on STDOUT:
NAME TIME DATE NUM FLOAT USD
----- -------- ---------- ----- ---------- --------
Bush 19:02:05 1998.09.13 12345 2345.2340 1,234.01
Hobbs 19:22:05 98.09.23 -1234 12345.2345 0.56
Here are a few examples of supported data types :
String
Integer
Integer
Integer
Float1
Float1
Float3
Time (HH:MM:SS)
String (HH > 24)
Date
Date (Year:MM:DD)
Date
Date
String (DD > 31)
Currency
Currency
Currency
String (leading group of digits longer than 3)
String (Decimal part is incomplete).
Float or Currency, depending on context.
Float or Currency, depending on context.
Usage: nosql listtotable [-t|--tabsize size] < table
Options:
Tabulator size. Any TAB characters in the data portion of input fields are replaced by size blank spaces (default is 4).
Converts a file in "list" format into a table. Long data fields may be folded over multiple lines in the input file, provided that each continuation line begins with one or more spaces and/or tabs.
Example:
Input file inputlist
:
NAME Bush
COUNT 44
TYP A
AMT 133
NAME Hansen
COUNT 44
TYP A
AMT 23
NAME Jones
COUNT 77
TYP X
AMT 77
NAME Perry
COUNT 77
TYP B
AMT 244
NAME Hart
COUNT 77
TYP D
AMT 1111
NAME Holmes
COUNT 65
TYP D
AMT 1111
Command:
nosql listtotable < inputlist
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql lock table [ table ... ]
Locks one or more tables by creating a lock (semaphore) file with the same name as the specified table, with the string ".LCK" appended. See section Data access control section further down in this manual for more details. This operator honours the shell variable NSQLOCKER, that can specify an alternative lockfile creation program to be used in place of the 'lock' script provided with NoSQL.
Example in the Bourne shell syntax:
if nosql lock table ; then .... do some modifications to table ... nosql unlock table else echo "The table is already locked" fi
Of course all programs must abide by this locking scheme for the
latter to be effective. Note that we must remove the lock explicitly
with nosql unlock table
afterwards, or the table will
remain unavailable (i.e. it will remain 'locked'). See the 'unlock'
operator further down in this manual.
Usage: nosql maketable < template
Builds a valid table header from a template file. Reads a template file from STDIN and prints the corresponding table header to STDOUT. A template file is very similar to a NoSQL 'list' file, except that it only contains column names, without the associated values, plus optional additional information like table documentation comments and column specific comments, i.e.:
# These are lines of table documentation. They can be of any length,
# and occur anywhere in the file, not just at the beginning.
# Each line must start correctly, e.g with "#", surrounded by any
# number of spaces and/or tabs.
Name Name of item
Type Type: 1,2,3,7,9,A,B,X
Count Number of items
K Constant modifier
SS7 Special status for type 7
Size In Kilobytes
Column names (on the left-hand side) must be separated from the associated optional comments by tabs and/or spaces. Comments (anything to the right of a column name) can span over multiple lines, provided that the first non-blank character of each continuation line be a hash (#).
If we feed the above template file to 'maketable' with:
nosql maketble < table.tpl
here is what we get on STDOUT:
Name Type Count K SS7 Size
---- ---- ----- - --- ----
The original file.tpl should be kept in the same directory as its associated table, for documentation.
Column names in the template file may optionally be preceeded by the column position number in the table header, i.e.:
# These are lines of table documentation. They can be of any length,
# and occur anywhere in the file, not just at the beginning.
# Each line must start correctly, e.g with "#", surrounded by any
# number of spaces and/or tabs.
1 Name Name of item
2 Type Type: 1,2,3,7,9,A,B,X
3 Count Number of items
4 K Constant modifier
5 SS7 Special status for type 7
6 Size In Kilobytes
Field numbers, if present, must be separated from column names by one or more spaces and/or tabs.
Usage: nosql merge [options] < old_table column ... merge_table
Options:
Add option. Add rows where the key column(s) match.
Delete option. Delete rows where the key column(s) match and the data value in the delete column is equal to the delete string, "..DEL.." (without the quotes) by default.
Like the delete option above but use 'STG' as the delete string.
Strip header from output.
Reverse option. The tables are sorted in reverse order.
Subset case. The structure of merge_table is a subset of the structure of old_table.
Debug option.
This operator merges and/or deletes rows of 'old_table' based on data values in 'merge_table' in the specified column(s). Both tables should be sorted on the specified column(s).
In the normal case, one or more rows in 'merge_table' either replace one or more existing rows in 'old_table' if the key column(s) match, or are inserted in order if the key column(s) do NOT match.
If the delete option is specified on the command line, one or more existing rows in 'old_table' will be deleted if there is a key column(s) match and the data in the delete column is equal to the delete string, "..DEL.." (without the quotes) by default. The delete column is the first non-key column in 'merge_table'.
Both tables should have similar data structures. The header for the new table is taken from 'merge_table', thus allowing a change of header information to be made.
This operator reads two tables, one from STDIN ad the other one from a file, and writes a table via STDOUT. Options may be abbreviated.
Usage: nosql mjoin [options] < table_1 column[=column_2] ... table_2
Options:
Do a cartesian (cross-product) join.
Do a "Master/Detail" join. The table from STDIN is the master.
Strip header from output.
Does a join of two tables on the (Key) column(s) specified. Unlikely
the standard join
, that operates on one single field per
input table, mjoin
can operate on multiple fields.
The disadvantage is that
mjoin
is slower than join
on large tables.
The default is a "natural" join, with optional "Master/Detail" or cartesian (cross-product) type joins.
Each item in the list of column name(s) specifys a key column, which may be different in the two tables, i.e. '=column_2', if given, refers to a name in table_2 that corresponds to 'column' in table_1. If '=column_2' is not given it means that the corresponding column name in both tables is the same.
If the key column names are different in the two tables, the name of the key columns in the output table will be from table_1.
Note that the two tables must be sorted on the key columns in order for a join operation to function correctly.
The order of columns in the output table will be: first the key columns, then the other columns from table_1, then the other columns from table_2.
This operator reads a table via STDIN and writes a table via STDOUT. Options may be abbreviated.
Usage: nosql modeof [-v|--verbose] file [ file ... ]
This operator takes a list of file names on the command line and prints the associated file permissions, in octal, to STDOUT. If '-v' is specified, then also the file name is printed to STDOUT, followed by the file permissions, with the two fields separated by a TAB character. If no files are specified then a usage message is printed to STDERR. If any of the files cannot be found, an error is printed to STDERR and a non zero exit code is returned.
Examples :
nosql modeof file1 file2 file3
0755
0600
0666
nosql modeof -v file1 file2 file3
file1 0755
file2 0600
file3 0666
Usage: nosql myprog [pathmode] yourprogram [<] table
where yourprogram is any program you like, followed by its
options and arguments, if any, and pathmode
can be
one of --path-first, --path-last, --path-asis
(default=--path-first
).
The path option can be specified in short
form, i.e. as -f, -l
and -a
respectively.
See explanation below.
Runs a local program under NoSQL. The program to be run is expected to be in a directory pointed to by the shell variable NSQMYLIB if defined, or $NSQLIB/mylib by default. The possibility of running local programs under NoSQL can be useful in many circumstances. Due to the simplicity of the NoSQL table data format, operators from other similar packages can be used in place of or in addition to the NoSQL ones. Suppose you are looking for a functionality that is not provided by any of the available operators, but which can be performed by an operator from RDB, Starbase or /rdb (running /rdb operators requires that you purchase the associated software license from RSW), then you can copy that program to $NSQLIB/mylib and use it seemlessly from within NoSQL with :
nosql myprog otherprogram ....
In alternative, you may write your own new operator and put it in $NSQLIB/mylib.
For example, suppose you want to run the Starbase column
operator in place of the NoSQL one, because the former is an
executable binary and sometimes executes faster. You can
copy it to $NSQLIB/mylib and run it just like any other NoSQL
program with:
nosql myprog column ....
This operator sets the program execution PATH in a way such that
$NSQLIB/mylib is searched first, followed by whatever was in your
PATH before. This is because sometimes there are operators which
name conflicts with system utilities. The Starbase 'column'
program conflicts with the /usr/bin/column
system program.
Putting $NSQLIB/mylib first in PATH ensures that
other Starbase
operators that want to use their 'column' program, will not pick the
system utility with the same name (which does a completely different
function). If for some reason you rather want $NSQLIB/mylib
to come last in PATH, you can use 'nosql myprog --path-last'
.
If you rather do not want that $NSQLIB/mylib appares in
PATH at all, you can use 'nosql myprog --path-asis'
.
Usage: nosql not command ...
Reverts the exit status of command
. If the command (or
operator) exit code is 0, it gets turned to 255. Conversely, if
it was non-zero it becomes 0. This can be useful for those
UNIX shells that do not understand the syntax 'if ! command
'.
For instance, if you want to test whether a file is a valid NoSQL table you would normally do :
if nosql istable < table ; then echo "The input file is a valid NoSQL table" else echo "The input file is NOT a NoSQL table" fi
You can revert the test in this way :
if nosql not istable < table ; then echo "The input file is NOT a NoSQL table" else echo "The input file is a valid NoSQL table" fi
Usage: nosql null [options] column ...
Options:
Return 0 if all the specified columns are null (default).
Return 0 if any of the specified columns are null.
Treat blanks as null.
Report null fields verbosely to STDERR.
Takes a NoSQL table on STDIN and checks the specified fields for null values. Returns 0 if the match is successful, or 255 otherwise. If any non-existent columns are specified on the command line, those columns will match the null value by definition, and the final result code will be either 0 or 255 depending on whether '--any' or '--all' (default) was specified. A column is considered to be null if all of its records are null.
Usage: nosql pick [-R|--raw-input] [-x|--hex] table < offsetlist
This operator takes a list of byte-offsets on STDIN and prints on
STDOUT the corresponding table records from the table specified
as a command line argument. If option '-x' is given, then the
offsets are expected to be base 16 numbers instead of the usual
base 10 numbers. The offsetlist
on STDIN can
be built with the 'index' operator, and 'pick' expects it to be
a one-column table, like this:
lpos
----
228
1117
518
1225
The above format can be obtained from a NoSQL secondary index file quite simply with the command :
nosql column lpos < index_table
The column name, 'lpos' in the example, can take any name, provided that the list has a valid NoSQL table format. If option '-R' is given, then the offsets are expected to be a blank- , tab- or newline-separated raw list of numbers on STDIN, i.e. they are not expected to to have a valid NoSQL table header.
Examples :
nosql column lpos < index_table | nosql pick table
or
echo "228 1117 518 1225" | nosql pick -R table
or even
cat <<_EOF_ | nosql pick -R table
228
1117 518
1225
_EOF_
This NoSQL operator uses the fseek(3) C library function and is extremely fast. It can be used to pick selected rows from a large, indexed table very quickly.
See also sect. index and search.
Usage: nosql pull table < keytable
This operator takes a list of table keys on STDIN and pulls
out the corresponding records from table
. It is quite fast
and can be used in place of 'nosql join' (see sect.
join) if we just need to get the foreign
key values of a relation between two tables. The records are pulled
from table
according to the key values specified in the
keytable
received on STDIN. The key coulmn(s) of interest
must be the first (leftmost) field in both table
and
keytable
. The two tables do not need to be sorted on the
key field for 'pull' to work. All 'pull' does is building the
proper command for egrep(1), so its speed is that of the
latter utility. Although egrep performs a linear scan of
table
, it has proven extremely fast, even for very
large tables (several megabytes in size, on a P-II 233 MHz CPU).
The 'pull' operator stores the whole egrep search pattern in
a shell variable. The UNIX shell used will limit the size of the
egrep pattern, which implies a limit on the number of key items
in keytable
.
Examples :
Suppose that table
contains the following data :
name nr typ amt
---- -- --- ---
Bush 1 A 133
Hansen 3 A 143
Hobbs 5 B 144
Jones 6 C 155
Perry 8 D 311
while keytable
contains :
key other
---- -----
Bush AA
Perry 123B
the command
nosql pull < keytable table
will produce the folowing output :
name nr typ amt
---- -- --- ---
Bush 1 A 133
Perry 8 D 311
Note how the key (leftmost) columns in table
and
keytable
need not be the same. The data rows and the output
column names will be those of table
.
Usage: nosql print [options] < table
Options:
By default, when a multi-line record of output for each row is necessary (due to the width of the current window or terminal) the program will try to fill space at the end of lines that would otherwise be wasted by moving some columns. This option prevents the moving of any columns.
This option attempts a "best fit" by rearranging columns (widest columns first). If 'N' is given the first N columns of the first line will not be moved.
Handle very large data fields, e.g. over 1000 chars. This option takes longer but it works for any size data fields.
Fold long data fields into multi line data based on field width. May be used with the '-t' option to limit the field width. Only a single line record of output is produced with this option.
Indent size of N spaces on 2nd and later lines of a multi-line record of output. Default is 4 spaces.
Line length of N chars for output. Default is the width of the current window or terminal.
Page size is of N lines. Default is the height of the current window or terminal. A value of zero '-p0' will turn paging off.
Page headings and settings for printing. A two line heading is put onto each page: page number, current date, and an optional string (stg). Sets page length (in lines) and line length (in chars) according to the value of 'X' as follows:
X: P page: 60 line: 80 (default font size)
X: R page: 47 line: 116 (rotated default)
X: A page: 51 line: 125 (rotated 10 point font)
X: 8 page: 63 line: 144 (rotated 8 point font)
X: 6 page: 82 line: 192 (rotated 6 point font)
X: W page: and line: from current window size.
Other desired page and/or line size options may be set after this in the option list.
Separator 'K' (which may be multi char) placed between columns. Default is two spaces.
Truncate data to the defined width. If N is given the width of printed fields will be limited to N chars.
List as many columns as possible in single line records that will fit in the current window or terminal width.
Debug mode.
This operator is used for quick and easy printing of a table, in a simple but useful form. Columns containing only numbers are right-justified with blanks, while anything else is left-justified.
The printing of each row of data will be on one line if possible, but when multiple lines are necessary the second and later lines are indented for readability. Also when multiple lines are necessary a simple space availability algorithm is used to minimize the number of lines printed for each row of data. This may result in the order of some data values being rearranged from their order in the table. The '-b0' option can override this algorithm and force the same printing order as in the table.
This NoSQL operator reads a table from STDIN and writes a formatted report on STDOUT. Options may be abbreviated.
As an example using the sample table from the DATA section (named sample), the command to view this table would be:
nosql print < sample
which would produce the output shown in Table 8. The same command with a page heading for printing:
nosql print -PP < sample
produces the output as shown in Table 9. Using a table (named sample4) that has long data values, shown in Table 10, the command to print the table using the truncate option is:
nosql print -t < sample4
Table 8
Printing table 'sample' using 'nosql print'
NAME COUNT TYP AMT
------ ----- ---- -----
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Table 9
Printing table 'sample' with page heading using 'nosql print'
Page 1 Mon Dec 2 16:56:43 PST 1991
NAME COUNT TYP AMT
------ ----- ---- -----
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Table 10
Table with long data values (sample4) actual content
name<T>type<T>contact<T>contents<N>
10<T>4<T>21<T>20<N>
Hansen<T>AAA<T>R. Starr at the UCLA & USC<T>Duplicate data under
processing order number 55-7.<N>
Hart<T>CCC<T>Hobbs/Emerson at RAND Corporation<T>85 files, 2 per
day containing 12 and 24 hour reports.<N>
Hobbs<T>EEE<T>Marshall at Universal AFB<T>Original PAF messages.
Both sets are incomplete.<N>
Bush<T>KKK<T>General USAF personnel<T>Duplicate ATO messages,
incomplete.<N>
Lender<T>RRR<T>Army base in Nevada<T>Nothing.<N>
Emerson<T>UUU<T>Navy at Washington DC<T>More than we thought at
first.<N>
which will produce output with the data values truncated to the defined column width as in Table 11. Using the same table with the fold option:
nosql print --fold < sample4
produces output with the long data values 'folded' within their defined column widths as shown in Table 12. Note that each line is repeated until the entire data value for each column is completely shown. This makes this type of output variable length.
If you need a quick and easy way to look at the data in an table use the -window option. This option will cause 'print' to list as many columns as possible in single line records that will fit in the current window or terminal width. Note that you do not have to type the column names (or even know them) to use this option.
Table 11
Printing table 'sample4' with 'nosql print --truncate' option
name type contact contents
---------- ---- --------------------- --------------------
Hansen AAA R. Starr at the UCLA Duplicate data under
Hart CCC Hobbs/Emerson at RAND 85 files, 2 per day
Hobbs EEE Marshall at Universal Original PAF message
Bush KKK General USAF personne Duplicate ATO messag
Lender RRR Army base in Nevada Nothing.
Emerson UUU Navy at Washington DC More than we thought
Table 12
Printing table 'sample4' with 'nosql print --fold' option
name type contact contents
---------- ---- --------------------- --------------------
Hansen AAA R. Starr at the UCLA Duplicate data under
& USC processing order
number 55-7.
Hart CCC Hobbs/Emerson at RAND 85 files, 2 per day
Corporation containing 12 and 24
hour reports.
Hobbs EEE Marshall at Universal Original PAF
AFB messages. Both sets
are incomplete.
Bush KKK General USAF Duplicate ATO
personnel messages,
incomplete.
Lender RRR Army base in Nevada Nothing.
Emerson UUU Navy at Washington DC More than we thought
at first.
It may be combined with the -t option to increase the number of columns of data shown on each line at the expense of some column width.
For example the command 'nosql print < d11c
' on an 80
character wide window or terminal produces the following:
name count type amt n1 n3
------ --------------- ---- ----- ------------------------ ------------
n2 n4 n5
------------------------ ------------------------ ------------
n6 n7
------------ ------------
Bush 3 A 133 alpha22.307 117722
baker DDBBx17 other
124567 8GGXXH17
Hansen 39 A 23 beta222.307 117723
charlie DDBBx18 data
1239870 GGXXH17
Newton 8 E 8 gama22.333 117724
dog DDBBx19 exists
1239870 GGXXH17
Hobbs 42 B 144 delta3.3.118 117725
echo DDBBx20 here
1239870 GGXXH17
Hart 2 C 55 epslion33.118 117726
foxtrot DDBBx21 also
1239870 GGXXH17
This is readable, but not very nice to look at, and even worse if there are more columns. The command
nosql print -w < d11c
produces:
name count type amt n1 n3
------ --------------- ---- ----- ------------------------ ------------
Bush 3 A 133 alpha22.307 117722
Hansen 39 A 23 beta222.307 117723
Newton 8 E 8 gama22.333 117724
Hobbs 42 B 144 delta3.3.118 117725
Hart 2 C 55 epslion33.118 117726
Not all the data is listed, but the first few columns (sometimes the most important) are easier to view. The command
nosql print -w -t6 < d11c
shows even more of the data, at the expense of some data width:
name count type amt n1 n2 n3 n4 n5 n6 n7
------ ------ ---- ----- ------ ------ ------ ------ ------ ------ ------
Bush 3 A 133 alpha2 baker 117722 DDBBx1 other 124567 8GGXXH
Hansen 39 A 23 beta22 charli 117723 DDBBx1 data 123987 GGXXH1
Newton 8 E 8 gama22 dog 117724 DDBBx1 exists 123987 GGXXH1
Hobbs 42 B 144 delta3 echo 117725 DDBBx2 here 123987 GGXXH1
Hart 2 C 55 epslio foxtro 117726 DDBBx2 also 123987 GGXXH1
Usage: nosql pwcrypt8
Takes the environment variable PW_CLEAR and prints its encrypted equivalent to STDOUT, in the usual 13-byte format. Handles cleartext passwords of max. 8 characters.
For example, the command
PW_CLEAR="MyPasswd" nosql pwcrypt8
will print something like 'vAmi1OYRFzYMQ
' to STDOUT.
If PW_CLEAR is unset, then nothing will be printed.
This operator can be useful to manage files with user IDs and passwords, stored in NoSQL format.
Usage: nosql pwmatch8 [-v|--verbose]
Takes two environment variables, PW_CLEAR and PW_CRYPT, and checks whether the former is the plaintext version of the latter. Returns 0 if it is, or 255 if it is not.
Handles cleartext passwords of max. 8 characters, while PW_CRYPT must be in the usual 13-byte format. If either variables are longer they will be truncated to fit.
For example, the command
PW_CLEAR="MyPasswd" PW_CRYPT="1tsMvBcaBkx0E" nosql pwmatch8
will return 0 if PW_CRYPT is the encrypted version of PW_CLEAR, or 255 otherwise.
This operator can be useful to manage files with user IDs and passwords, stored in NoSQL format.
Usage: nosql record from [to] < table
Takes a record number and outputs the corresponding row of the input table. If two numbers are specified, then outputs the corresponding range of records. If the second number is lesser than the first, then all records starting from the first record to the end of table are printed. If no record or if an out-of-range record is specified, then only the table header is printed.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql record 3 5 < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Usage: nosql rename oldname newname < table
Renames a specified column. This operator takes exactly two arguments: 'oldname' and 'newname'. If more arguments are specified, they are silently disregarded. If neither 'oldname' nor 'newname' are specified, then the input table is simply printed to STDOUT unchanged. The same applies if only one name is specified.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql rename COUNT NUMBER < inputtable
Output on STDOUT:
NAME NUMBER TYP AMT
---- ------ --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql repair [-F|--filler string] < table
Appends missing trailing tabs to data rows of a table. Takes table rows that may lack trailing empty fields, and pads them with TAB characters up to the No. of fields in the header. Such broken tables are often the result of manipulating tables with common spreadsheet programs. If option '-F string' is specified, then padding is done using whatever specified by string instead of using NULL fields. This will make added fields easier to spot visually.
To check whether a table is broken, the NoSQL operator see
(explained further down) may be used:
Input file brokentable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X
Perry 77 B 244
Hart 77 D
Holmes 65 D 1111
There are missing values in column AMT, but this could be ok if the corresponding TAB characters are there. To check that out we do :
nosql see < brokentable
and what we get on STDOUT is :
NAME^ICOUNT^ITYP^IAMT$
----^I-----^I---^I---$
Bush^I44^IA^I133$
Hansen^I44^IA^I23$
Jones^I77^IX$
Perry^I77^IB^I244$
Hart^I77^ID^I$
Holmes^I65^ID^I1111$
the see
operator represents TAB characters with '^I'
and newlines with '$'. As we can see, there is a missing TAB
at the end of line 3 (Jones), while at line 5 the missing data value
is correctly replaced by a TAB (followed by the trailing newline).
To repair the table we do :
nosql repair < brokentable | nosql see
And what we now get on STDOUT is :
NAME^ICOUNT^ITYP^IAMT$
----^I-----^I---^I---$
Bush^I44^IA^I133$
Hansen^I44^IA^I23$
Jones^I77^IX^I$
Perry^I77^IB^I244$
Hart^I77^ID^I$
Holmes^I65^ID^I1111$
where line 3 (Jones) is now fixed.
Usage: nosql rmcol columnname ... < table
Removes the specified column(s) from a table. Nonexisting columns specified on the command line are silently ignored. If only invalid columns are specified, then nothing is deleted.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql rmcol TYP COUNT < inputtable
Output on STDOUT:
NAME AMT
---- ---
Bush 133
Hansen 23
Jones 77
Perry 244
Hart 1111
Holmes 1111
Usage: nosql row [-x|--debug]'selection expression' < table
Options:
Debug option. Print the actual AWK program to STDERR before executing it.
Selects rows from the input table based on an arbitrary AWK expression using column names. Characters that are special to the UNIX shell must be quoted. The safest bet is to enclose the whole awk expression in single quotes, to prevent the shell from parsing it.
If the input table contains duplicated columns, with the same name but different values, only the first (leftmost) one is taken into account, but the output table will still have the duplicates.
This operator reads a table via STDIN and writes a table via STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql row 'NAME=="Bush"' < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Comments, i.e. unquoted # signs in the AWK expression are not supported. Pattern matching inside the expression should follow the same rules explained in the section about the compute operator.
Usage:
nosql search [options] inputtable < keytbl
or:
nosql search [options] -ind index_file [inputtable] < keytbl
Options:
Index file search.
Strip header from output.
Partial (initial) match. Applies to string type data only.
Only a single row match is needed.
Reverse sort option. Input table is sorted in reverse order.
Verify-only mode. If an item of info from keytbl is valid prints "ok", else an error message, on STDERR. NO new table is produced. Used by other processes for verification.
Debug option.
This operator does a fast search of 'inputtable' (or index_file) using a binary search on a key of one or more columns. The table (or index_file) must be sorted on the key columns. Each column in the key may be of type string or type numeric (but be carefull with numeric data and exact matches). In the second form of usage for this operator if 'inputtable' is not given its name will be inferred from the name of index_file. For example if index_file is 'skb.x.typ' then the table name inferred will be 'skb.rdb'.
The column in the file 'keytbl' specifies both the key column name and the argument values to search for. File 'keytbl' is in the usual NoSQL table format.
Normally an argument value and a data field must compare exactly for a match to occur (exact match). If the paritial match otpion '-p' is selected, and if the argument value compares with the initial part of the data field it is considered a match. This applies to string type data only. Note that for numeric type data an exact match is always necessary.
Normally all rows that match will be written to the new table, in the same order as in the old table. If only a single row key match is appropriate some execution time can be saved by specifing the '-sgl' option.
This operator writes a table via STDOUT. Options may be abbreviated. Returns the number of non-finds at exit.
Usage: nosql see < table
TAB and newline characters that define a structure of a NoSQL
table are normally not visible. Sometimes, especially for large
tables, it is not easy to look for missing or extra tabs that
make a table invalid. This operator is simply a call to the
UNIX command 'cat -vte
'. This command shows TAB characters
as `^I' and newlines as '$', making it much easier to see what is
wrong in the broken table.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql see < inputtable
Output on STDOUT:
NAME^ICOUNT^ITYP^IAMT$
----^I-----^I---^I---$
Bush^I44^IA^I133$
Hansen^I44^IA^I23$
Jones^I77^IX^I77$
Perry^I77^IB^I244$
Hart^I77^ID^I1111$
Holmes^I65^ID^I1111$
Usage: nosql setfirst [-S|--subsep string] columnname < table
Moves a specified column to the first position in table. Takes a specified column and moves it to the first (leftmost) position in table, while the column that was in the first position is exchanged with the former. If an invalid target column is specified, then the input table is printed unchanged to STDOUT. The same happens if no target column is specified. If option '-S string' is specified, then the column moved to the 1st position is then 'normalized' as a potential key column. Normalization consists of stripping leading and trailing blanks, and replacing multiple middle blanks between words with the sub-field separator specified by 'string'. This will usually be one single character, like a colon (:), a semicolon (;), a slash (/), etc., but it must not contain TAB nor newline characters, or the resulting output table will be broken. Characters that are special to the Unix shell must be quoted or escaped in 'string'.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql setfirst TYP < inputtable
Output on STDOUT:
TYP COUNT NAME AMT
--- ----- ---- ---
A 44 Bush 133
A 44 Hansen 23
X 77 Jones 77
B 77 Perry 244
D 77 Hart 1111
D 65 Holmes 1111
Usage: nosql shelltotable < varlist
Converts a list of shell variable assignments into a one-record table. Converts a list of shell variable assignments (VARIABLE=value), with multiple statements separated by newlines, into a NoSQL table.
This operator reads a newline separated list of VARIABLE=value pairs from STDIN and produces the corresponding table to STDOUT.
Example:
Command:
env | grep ^H | nosql shelltotable
Output on STDOUT:
HOSTNAME HOSTTYPE HOME
-------- -------- ------------
tango i486 /home/carlos
Usage: nosql show [c|w]
Shows NoSQL copyright terms (option c) and writer's information (option w), as suggested by the text of the GNU General Public License.
Usage: nosql sort [options] [column] ... < table
Options:
Debug option. Print the actual UNIX sort command line to STDERR.
Additional sort options that can be specified on the command line are '-cbdfinru'. See sort(1) for more info on what each option means.
Sorts a table on one or more columns, using the UNIX 'sort' utility. This operator reads a table via STDIN and writes a table via STDOUT. If no sort columns are specified, then the input table is sorted on entire rows. If nonexistent columns are specified, they are silently ignored.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql sort -n -r NAME AMT < inputtable
Output on STDOUT:
NAME COUNT TYP AMT
---- ----- --- ---
Holmes 65 D 1111
Hart 77 D 1111
Perry 77 B 244
Bush 44 A 133
Jones 77 X 77
Hansen 44 A 23
Usage: nosql soundex [-n|--name name] column [table]
Compute the soundex codes for a table column. Takes a NoSQL table and prints the soundex code associated with each data value in a specified column.
This operator reads a table from STDIN and prints a new table to STDOUT. The output table is the input table with an added leading column (the 'soundex' column). The new column contains the soundex code of the data in 'column'. The new column name is the same as 'column', with an "s" prepended to it, or the name specified with option '-n' if present. For instance, if the requested column is 'Name', then the computed soundex column default name will be 'sName'.
Example :
input file table
NAME COUNT TYP AMT
---- ----- --- ---
Holmes 65 D 1111
Hart 77 D 1111
Perry 77 B 244
Bush 44 A 133
Jones 77 X 77
Hansen 44 A 23
nosql soundex NAME < table
Output table :
sNAME NAME COUNT TYP AMT
----- ---- ----- --- ---
N500 Holmes 65 D 1111
Z000 Hart 77 D 1111
H452 Perry 77 B 244
H630 Bush 44 A 133
P600 Jones 77 X 77
B200 Hansen 44 A 23
Usage: nosql split [-o|--output] [headerfile] < table
Separates the header from the body of a table.
Splits a table apart, separating the header section from the body.
The header data is written to whatever file was specified on the
command line, or to /dev/null
if not specified.
This NoSQL operator reads a table from STDIN and writes the table header to a file and the table body to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql split headfile < inputtable
Output on STDOUT:
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
cat headfile
NAME COUNT TYP AMT
---- ----- --- ---
Usage: nosql subtotal [options] B_column ... [-s|--subtotal] column ...
Options:
Include totals.
Total Only Option. Just do totals on specified columns. In this case the '-s' flag is not necessary.
This operator lists subtotals of specified column(s) whenever the value of specified break columns(s) (B_column(s)) changes. A single break column will produce a sub-total of all specified columns on each line. If there is more than one break column given then in addition whenever the value of the first break column changes an additional line will be printed showing the sub-total for that group.
If no break column is given the first column is used; if no subtotal column is given then all columns of type numeric are sub-totaled.
This operator reads a table via STDIN and writes a table via STDOUT. Options may be abbreviated.
Example table (named small
):
name amt typ count n
---- --- --- ----- -
Hansen 39 A 23 3
Hansen 9 A 3 3
Hansen 9 B 3 4
Jones 42 B 144 5
Jones 4 B 14 5
Hart 9 C 3 5
Hart 2 C 55 6
Hart 2 D 55 6
Hobbs 57 X 7 4
Hobbs 5 X 57 4
The output from the command:
nosql subtotal name -s amt < small | nosql print
would be:
name amt
------ ---
Hansen 57
Jones 46
Hart 13
Hobbs 62
The output from the command:
nosql subtotal name typ -s amt count < small | nosql print
is shown in Table 18.
Table 18
Output from the nosql 'subtotal' operator
name typ amt count
------ --- --- -----
Hansen A 48 26
Hansen B 9 3
57 29
Jones B 46 158
46 158
Hart C 11 58
Hart D 2 55
13 113
Hobbs X 62 64
62 64
Usage: nosql summ [options] [column ...]
Options:
A Count of the unique values for each column given.
Like option '-cu' but also shows counts of null (empty) and blank values (have only space chars), if either exist.
A Count of each unique value for each column given.
Like option '-cuu' but shows only counts greater than one.
The min, average, max, total for each column given.
Inverse option. Selects all columns except those named.
Produces "summary" information about a table. If no columns are given then information about all columns is produced. A Count of the data rows is always shown.
This operator reads a table via STDIN and writes a summary report via STDOUT. Options may be abbreviated.
Usage: nosql tabletolist < table
Converts a table to "list" format. This NoSQL operator reads a table from STDIN and converts it to "list" format on STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql tabletolist < inputtable
Output on STDOUT:
NAME Bush
COUNT 44
TYP A
AMT 133
NAME Hansen
COUNT 44
TYP A
AMT 23
NAME Jones
COUNT 77
TYP X
AMT 77
NAME Perry
COUNT 77
TYP B
AMT 244
NAME Hart
COUNT 77
TYP D
AMT 1111
NAME Holmes
COUNT 65
TYP D
AMT 1111
Usage: nosql tabletoshell [options] < table
Options:
Simply truncate the 'value' part of each assignment at the first occurrence of the single-quote char ('), instead of translating the latter to '''. See explanations below.
Prefix each output variable (column) name with string 'P'.
Converts a single-record table to shell variable format (VARIABLE='value'), handy for grabbing a table row into a shell program. If the table contains more than one row of data, then the assignments will be those from the last row. Any single-quotes in the 'value' part of the assignment are escaped with the ASCII sequence ''', not to cause troubles to the invoking shell, or they are used simply as upper boundary of the 'value' part of the assignment if '-t' is specified.
This operator reads a table from STDIN and prints the corresponding shell variable assignments to STDOUT.
Example of usage from within a shell script:
eval `nosql tabletoshell < table`
Usage: nosql template < table
Builds a table template file for the input NoSQL table.
This NoSQL operator reads a table from STDIN and prints the corresponding table template file to STDOUT, in a format suitable for 'maketable'.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql template < inputtable
Output on STDOUT:
1 NAME
2 COUNT
3 TYP
4 AMT
Usage: nosql trim [-m|--strip-all] [-e|--empty-rows] < table
Removes surrounding blanks from column values. If option '-m' is specified then multiple blanks between the words are stripped to one single blank. If option '-e' is specified then empty rows are removed from the input table.
Usage: nosql ucfirst < table
Turns to lower-case the column names and capitalizes the first letter of each one, to improve column name readability.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT.
Example:
Input table inputtable
:
NAME COUNT TYP AMT
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Command:
nosql ucfirst < inputtable
Output on STDOUT:
Name Count Typ Amt
---- ----- --- ---
Bush 44 A 133
Hansen 44 A 23
Jones 77 X 77
Perry 77 B 244
Hart 77 D 1111
Holmes 65 D 1111
Usage: nosql uniq [options] column ... < table
Options:
Diagnostic output. Prints number of rows removed on STDERR.
Group option. Instead of producing unique rows, produce only groups of repeated rows, with a blank row between. In this case the '-D' option shows the number of groups.
Inverse option. Selects all columns except those named.
Reads the input table and compares adjacent rows. The second and succeeding copies of repeated rows, considering only the selected columns, are removed. That is, adjacent rows are considered equal if the data values in all of the selected columns are equal. The remaining rows are written to STDOUT.
Note that repeated rows must be adjacent in order to be found. Normally this means that the input table should be sorted on the selected columns for this capability to work properly.
This NoSQL operator reads a table from STDIN and writes a table to STDOUT. Options may be abbreviated.
Usage: nosql unlock table [table ...]
Removes the lock file(s) created by 'nosql lock' against the same table(s).
Usage: nosql version
Prints the NoSQL version and a short copyright notice to STDOUT.
Usage: nosql weed [-o|--output] [weedfile] < table
Removes invalid records from a table.
Discards any records whose number of TAB-separated fields does
not match the number of fields in the table header. The failing rows
are optionally written to the file specified on the command line,
or to /dev/null
if no output file is specified.
This NoSQL operator reads a table from STDIN and writes the purified table to STDOUT.
Usage: nosql whatis [string] Displays a short description of all the NoSQL operators which name contains string.
Usage: nosql write [options] [-o|--output] outputtable < inputtable
Options:
Do not regard the input file as a NoSQL table, but rather as a generic, unstructured file. This must be used when the input file lacks a valid NoSQL header, or the write operation will fail.
Suppress writing the table to STDOUT.
Keep the table sorted on field when writing the input table back to disk. If unspecified, the table is kept sorted on the first (leftmost) field by default.
Besides sorting on the key field, the output table is also kept unique on the same field. Rows that are duplicated on that field will be silently dropped. This option should be used with care.
This option does nothing. It has been added for better compliancy with the GNU coding standards.
Reads a table via STDIN and [over]writes the table given as a command line argument, printing the output table also to STDOUT. The function of this operator is somewhat similar to that of the UNIX utility tee(1).
This operator can be safely used in constructs like :
nosql compute ... < table | nosql write -o table | ...
without worrying that the original input table be destroyed before the first pipeline has completed.
If no output stream is desired on STDOUT then the latter
can be directed to /dev/null
, as usual, or the '-s' command
line switch may be used.
The table may be an existing file, or it may be automatically checked out from RCS. In the latter case it will be checked back into RCS after the operation is complete. The default action is that if the table does not exist an attempt will be made to find the table under RCS.
To start RCS versioning on a table for the first time, the following command can be used:
ci [-u] table
From now on, write
will handle all interactions with
RCS automatically.
WARNING: 'write' does not do any locking on the output table. Any locking, if necessary, must be handled by the invoking program.
Example of usage from within a Bourne shell script, with locking :
if nosql lock outputtable then nosql compute ... < inputtable | nosql write -s -o outputtable nosql unlock outputtable fi
where inputtable
and outputtable
can be the same.
This fragment of code ensures that either the lock can be set, or no modifications to 'outputtable' will be done.
This operator is especially meant to be used in programs that need to modify tables 'in place'. It can optionally take a backup copy of the target table, which will have the same name as the destination table, prefixed by a comma (,).