enoch/n2n
1
0
Fork 0
mirror of https://github.com/ntop/n2n.git synced 2024-09-20 00:51:10 +02:00
Code Issues Actions Packages Projects Releases 1 Wiki Activity
n2n/doc/Scripts.md

70 lines
2.0 KiB
Markdown
Raw Normal View History

Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
# Scripts
There are a number of useful scripts included with the distribution.
Some of these scripts are only useful during build and development, but
other scripts are intended for end users to be able to use. These scripts
may be installed with n2n as part of your operating system package.
Clearly separate documented scripts into user and build categories
2021-11-01 11:30:38 +01:00
All scripts can be found in the `scripts` directory.
Basic C Code lint checker and shell checker (#859) * Factor build packages out into a more maintainable list * Create a location for scripts to live * Provide a make target to return the source dir as close as reasonable to the original distributed state * Add a code lint step, checking the coding style * Change test harness as recommended by shellcheck * Ensure we actually have the linter tool installed * Use the correct directory for cmake to run the tests * Adjust for the older uncrustify in the current github ubuntu-latest * Make one file pass the linter * Integrate the lint with the existing test workflow * Add files with minimal changes needed to the linter * Add more files with minimal changes needed to the linter * Dont build binaries if we fail the lint test * Update the phony targets with the lint steps * Ensure the flake8 package is installed in the new lint workflow job * Use the makefile to drive the packages needed to install for linting * No need to add dependancies on lint, just rely on the workflow status to show failure * Update the scripts dir README to reflect current assumptions * Rename and briefly document the indent.sh script * Fix the ignore to ignore the right Makefile * Rename the test_harness script to make it clear it is a shell script * Provide a master lint make target and add a shell script lint tool * Elminate stray tabs * Drop include/auth.h from linter - there are inconsistant results with function definitions when using the current uncrustify rules
2021-10-23 21:36:18 +02:00
Clearly separate documented scripts into user and build categories
2021-11-01 11:30:38 +01:00
Short descriptions of these scripts are below.
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
Clearly separate documented scripts into user and build categories
2021-11-01 11:30:38 +01:00
## End user scripts
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
Clearly separate documented scripts into user and build categories
2021-11-01 11:30:38 +01:00
### `n2n-ctl`
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
This python script provides an easy command line interface to the running
JSON Reply Management API - feature parity with old management interfaces (#861) * Ensure that recent code additions pass the linter * Include some of the more obviously correct lint fixes to edge_utils.c * Refactor edge JSON api into its own source file * Use shorter names for static management functions * Implement a JSON RPC way of managing the verbosity * Tidy up help display in n2nctl script * Make note of issue with implementing the stop command * Implement a JSON RPC call to fetch current community * Make n2nhttpd time value be more self-contained * Make n2nhttpd order more closely match the existing management stats output * Wire up status page to the verbosity setting * Add JSON versions of the remainder of the edge management stats * Add new file to cmake * Properly define management handler * Only update the last updated timestamp after a successful data fetch * Function and types definition cleanup * Force correct type for python scripts mgmt port * Implement initial JSON API for supernode * Fix whitespace error * Use helper function for rendering peers ip4 address * Proxy the auth requirement back out to the http client, allowing normal http auth to be used * Ensure that we do not leak the federation community * Use the same rpc method name and output for both edge and supernode for peers/edges * Allow n2nctl to show raw data returned without resorting to tricks * Make n2nctl pretty printer understandable with an empty table * Use the full name for supernodes RPC call * Use same RPC method name (but some missing fields) for getting communities from both edge and supernode * Add *_sup_broadcast stats to edge packet stats output * Refacter the stats into a packetstats method for supernode RPC * Even if I am not going to prettyprint the timestamps, at least make all the timestamps on the page the same unit * Simplify the RPC handlers by flagging some as writable and checking that in the multiplexer * Remove invalid edges data * Avoid crash on bad data to verbose RPC * Avoid showing bad or inconsistant protocol data in communities RPC * Minor clarification on when --write is handled * Make linter happy * Fix changed method name in n2nhttpd * Move mainloop stop flag into the n2n_edge_t structure, allowing access from management commands * Implement edge RPC stop command * Move mainloop stop flag into the n2n_sn_t structure, allowing access from management commands * Implement supernode RPC stop command * Allow multiple pages to be served from mini httpd * Extract common script functions into a separate URL * Handle an edge case in the python rpc class With a proper tag-based demultiplexer, this case should be a nop, but we are single-threaded and rely on the packet ordering in this library. * Add n2nhttpd support to query supernode using urls prefixed with /supernode/ * Handle missing values in javascript table print * Add another less filtering javascript key/value renderer * Add a supernode.html page to the n2nhttpd * Address lint issue * Mention the second html page on the Scripts doc * Remove purgable column from supernode edges list - it looks like it is rarely going to be set * Add a simple one-line example command at the top of the API documentation * Acknowledge that this is not the most efficient protocol, but point out that it was not supposed to be * Make it clear that the n2nctl script works for both edge and supernode * Fight with inconsistant github runner results * Turn off the /right/ coverage generator
2021-10-23 07:20:05 +02:00
n2n processes. It uses UDP communications to talk to the Management API.
By specifying the right UDP port, it can talk to both the edge and the
supernode daemons.
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
Example:
Rename scripts for better consistancy (#866) * Address shellcheck concerns with n2n_gateway.sh script * Rename n2n_gateway.sh to live with all the other scripts (and update docs) * Rename hack_fakeautoconf to make it clearly a shell script * Address shellcheck concerns with hack_fakeautoconf.sh * Rename python scripts to match other n2n tools * Fix windows compile - when streamlining the use of hack_fakeautoconf.sh, I forgot to update all uses of this script
2021-10-24 00:13:01 +02:00
- `scripts/n2n-ctl --help`
- `scripts/n2n-ctl help`
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
Clearly separate documented scripts into user and build categories
2021-11-01 11:30:38 +01:00
### `n2n-httpd`
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
This python script is a simple http gateway to the running edge. It provides
a proxy for REST-like HTTP requests to talk to the Management API.
By default it runs on port 8080.
JSON Reply Management API - feature parity with old management interfaces (#861) * Ensure that recent code additions pass the linter * Include some of the more obviously correct lint fixes to edge_utils.c * Refactor edge JSON api into its own source file * Use shorter names for static management functions * Implement a JSON RPC way of managing the verbosity * Tidy up help display in n2nctl script * Make note of issue with implementing the stop command * Implement a JSON RPC call to fetch current community * Make n2nhttpd time value be more self-contained * Make n2nhttpd order more closely match the existing management stats output * Wire up status page to the verbosity setting * Add JSON versions of the remainder of the edge management stats * Add new file to cmake * Properly define management handler * Only update the last updated timestamp after a successful data fetch * Function and types definition cleanup * Force correct type for python scripts mgmt port * Implement initial JSON API for supernode * Fix whitespace error * Use helper function for rendering peers ip4 address * Proxy the auth requirement back out to the http client, allowing normal http auth to be used * Ensure that we do not leak the federation community * Use the same rpc method name and output for both edge and supernode for peers/edges * Allow n2nctl to show raw data returned without resorting to tricks * Make n2nctl pretty printer understandable with an empty table * Use the full name for supernodes RPC call * Use same RPC method name (but some missing fields) for getting communities from both edge and supernode * Add *_sup_broadcast stats to edge packet stats output * Refacter the stats into a packetstats method for supernode RPC * Even if I am not going to prettyprint the timestamps, at least make all the timestamps on the page the same unit * Simplify the RPC handlers by flagging some as writable and checking that in the multiplexer * Remove invalid edges data * Avoid crash on bad data to verbose RPC * Avoid showing bad or inconsistant protocol data in communities RPC * Minor clarification on when --write is handled * Make linter happy * Fix changed method name in n2nhttpd * Move mainloop stop flag into the n2n_edge_t structure, allowing access from management commands * Implement edge RPC stop command * Move mainloop stop flag into the n2n_sn_t structure, allowing access from management commands * Implement supernode RPC stop command * Allow multiple pages to be served from mini httpd * Extract common script functions into a separate URL * Handle an edge case in the python rpc class With a proper tag-based demultiplexer, this case should be a nop, but we are single-threaded and rely on the packet ordering in this library. * Add n2nhttpd support to query supernode using urls prefixed with /supernode/ * Handle missing values in javascript table print * Add another less filtering javascript key/value renderer * Add a supernode.html page to the n2nhttpd * Address lint issue * Mention the second html page on the Scripts doc * Remove purgable column from supernode edges list - it looks like it is rarely going to be set * Add a simple one-line example command at the top of the API documentation * Acknowledge that this is not the most efficient protocol, but point out that it was not supposed to be * Make it clear that the n2nctl script works for both edge and supernode * Fight with inconsistant github runner results * Turn off the /right/ coverage generator
2021-10-23 07:20:05 +02:00
It also provides a simple HTML page showing some edge information, which when
run with default settings can be seen at http://localhost:8080/ (Also
a http://localhost:8080/supernode.html page for the supernode)
Cleanup and Documentation for JSON management API (#856) * Reimplement JSON mgmt with clear separation of read/write actions * Reduce boilerplate by using a table driven command definition for json mgmt commands * Port tools to use new json api * Add a basic authentication for json mgmt commands * If a auth key is given, it must match * Add auth key to management scripts * Add a flag bitfield to clearly turn the tag param into a options list * Allow simple pass-through of any command from n2nctl * Convert the n2nctl to use an object oriented interface * Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely * Remove some repetition from the server * Use the correct options to allow reuseaddr * Dont generate a scary message on ctrl-c * Convert n2nhttpd to use object based RPC * Use the same longopt for both tools * Pass any extra args through to the RPC * Add some documentation for the scripts in the repository * Spelling fix * Add documentation for the JSON reply mangement API
2021-10-17 22:16:42 +02:00
Example:
Rename scripts for better consistancy (#866) * Address shellcheck concerns with n2n_gateway.sh script * Rename n2n_gateway.sh to live with all the other scripts (and update docs) * Rename hack_fakeautoconf to make it clearly a shell script * Address shellcheck concerns with hack_fakeautoconf.sh * Rename python scripts to match other n2n tools * Fix windows compile - when streamlining the use of hack_fakeautoconf.sh, I forgot to update all uses of this script
2021-10-24 00:13:01 +02:00
- `scripts/n2n-httpd --help`
- `scripts/n2n-httpd 8087`
Clearly separate documented scripts into user and build categories
2021-11-01 11:30:38 +01:00
## Build and Development scripts
### `hack_fakeautoconf.sh`
This shell script is used during development to help build on Windows
systems. An example of how to use it is shown in
the [Building document](Building.md)
### `indent.sh`
This shell script is a wrapper for the `uncrustify` C code style checker
which checks or applies a set of rules to the code. It is used during
the automated lint checks.
### `test_harness.sh`
This shell script is used to run automated tests during development.
### `n2n-gateway.sh`
Rename scripts for better consistancy (#866) * Address shellcheck concerns with n2n_gateway.sh script * Rename n2n_gateway.sh to live with all the other scripts (and update docs) * Rename hack_fakeautoconf to make it clearly a shell script * Address shellcheck concerns with hack_fakeautoconf.sh * Rename python scripts to match other n2n tools * Fix windows compile - when streamlining the use of hack_fakeautoconf.sh, I forgot to update all uses of this script
2021-10-24 00:13:01 +02:00
A sample script to route all the host traffic towards a remote gateway,
which is reachable via the n2n virtual interface.
Use script to calculate the build version
2021-11-01 11:39:25 +01:00
### `version.sh`
This script is used to determine the current version number during the
build process.
It looks at both the VERSION file and the GIT tags and outputs the
version number to use.
Reference in New Issue Copy Permalink