Commit Graph

138 Commits

Author SHA1 Message Date
Hamish Coleman
d81908d6a2 Move as much as simply possible from edge_management into shared code 2022-04-17 00:25:04 +01:00
Logan oos Even
f3e305b254
updated Faq.md 2022-04-11 11:14:21 +02:00
Logan oos Even
661858aee8
updated Scratchpad.md 2022-03-31 15:03:10 +02:00
Hamish Coleman
4b8aaa7f6e
Merge pull request #947 from hamishcoleman/json_pubsub
Add JSON pubsub framework for edge
2022-02-12 10:59:52 +00:00
Hamish Coleman
99d56a8a7b
Improve build docs (#952) 2022-02-09 21:30:26 +00:00
Hamish Coleman
a40ed15d93 Add pubsub system to edge, with one example TEST channel 2022-02-05 20:06:31 +00:00
Hamish Coleman
40773b4e4d Ensure we have a local copy of the tag in our request structure - which will be useful for the event framework 2022-02-05 20:06:31 +00:00
Hamish Coleman
ecaba980ba Attempt to simplify adding new tests 2022-02-05 20:06:31 +00:00
Hamish Coleman
e22e453d2c Add an integration test and a bunch of framework to support tests on multiple build systems 2022-02-05 20:06:31 +00:00
Hamish Coleman
7f163858cf
Update to Compiling documentation (#945) 2022-01-30 17:39:10 +00:00
Hamish Coleman
028d6f9b07
Harmonise configuration defaults (#937)
* Ensure all options start off for cmake too

* Attempt to make the build time config logic in the cmake process match the makefile process

* Add a simple muscle memory helper for cmake

* Ask them what they are smoking, then tell them to put it out

* Avoid occasional stale package cache errors
2022-01-25 20:11:02 +00:00
Hamish Coleman
45520f3d24
Move Build config docs into its own document (#928) 2022-01-10 22:23:37 +00:00
Hamish Coleman
da9ba27b0a
Minor build document improvments (#926)
* Improve some build documentation

* Address build failure
2022-01-09 18:07:29 +00:00
skyformat99
bc4789ae0e
updated building documentation (#911)
* Update README.md

update readme.md

* moved detailed content to doc folder

Co-authored-by: Logan oos Even <Logan.00sEven@gmail.com>
2021-12-24 20:33:39 +01:00
Hamish Coleman
f5e2f3086e
Add munin monitoring plugin (#892)
* Add munin monitoring plugin

* Ensure stable field ordering for munin
2021-11-05 11:14:47 +00:00
Hamish Coleman
a38ec0f033 Update windows cmake build docs to match current process 2021-11-02 09:13:12 +00:00
Hamish Coleman
aafca3813a Use script to calculate the build version 2021-11-01 19:15:23 +00:00
Hamish Coleman
d57ac3c6a1 Clearly separate documented scripts into user and build categories 2021-11-01 19:15:23 +00:00
Logan oos Even
3d0c618d99
updated Federation.md 2021-11-01 17:20:09 +05:45
Logan oos Even
14c881928a
updated Building.md 2021-10-31 15:40:23 +05:45
Logan oos Even
9c50e96e7b
updated Authentication.md 2021-10-26 03:31:01 +05:45
Logan oos Even
1690241b9b
updated Scratchpad.md 2021-10-25 02:04:57 +05:45
Logan oos Even
3b187b4ac8
updated JSON API password handling '--management-password <pw>' (#869) 2021-10-24 15:22:36 +05:45
Logan oos Even
7bace3755c
updated Federation.md 2021-10-24 13:09:33 +05:45
Logan oos Even
c61b62ab1b
choose supernode selection strategy at run-time '--select-rtt' (#864) 2021-10-24 12:53:25 +05:45
Hamish Coleman
7eb3f2cd7f
Rename scripts for better consistancy (#866)
* Address shellcheck concerns with n2n_gateway.sh script

* Rename n2n_gateway.sh to live with all the other scripts (and update docs)

* Rename hack_fakeautoconf to make it clearly a shell script

* Address shellcheck concerns with hack_fakeautoconf.sh

* Rename python scripts to match other n2n tools

* Fix windows compile - when streamlining the use of hack_fakeautoconf.sh, I forgot to update all uses of this script
2021-10-24 03:58:01 +05:45
Hamish Coleman
80b33cd1a9
Basic C Code lint checker and shell checker (#859)
* Factor build packages out into a more maintainable list

* Create a location for scripts to live

* Provide a make target to return the source dir as close as reasonable to the original distributed state

* Add a code lint step, checking the coding style

* Change test harness as recommended by shellcheck

* Ensure we actually have the linter tool installed

* Use the correct directory for cmake to run the tests

* Adjust for the older uncrustify in the current github ubuntu-latest

* Make one file pass the linter

* Integrate the lint with the existing test workflow

* Add files with minimal changes needed to the linter

* Add more files with minimal changes needed to the linter

* Dont build binaries if we fail the lint test

* Update the phony targets with the lint steps

* Ensure the flake8 package is installed in the new lint workflow job

* Use the makefile to drive the packages needed to install for linting

* No need to add dependancies on lint, just rely on the workflow status to show failure

* Update the scripts dir README to reflect current assumptions

* Rename and briefly document the indent.sh script

* Fix the ignore to ignore the right Makefile

* Rename the test_harness script to make it clear it is a shell script

* Provide a master lint make target and add a shell script lint tool

* Elminate stray tabs

* Drop include/auth.h from linter - there are inconsistant results with function definitions when using the current uncrustify rules
2021-10-24 01:21:18 +05:45
Hamish Coleman
ae502d9181
JSON Reply Management API - feature parity with old management interfaces (#861)
* Ensure that recent code additions pass the linter

* Include some of the more obviously correct lint fixes to edge_utils.c

* Refactor edge JSON api into its own source file

* Use shorter names for static management functions

* Implement a JSON RPC way of managing the verbosity

* Tidy up help display in n2nctl script

* Make note of issue with implementing the stop command

* Implement a JSON RPC call to fetch current community

* Make n2nhttpd time value be more self-contained

* Make n2nhttpd order more closely match the existing management stats output

* Wire up status page to the verbosity setting

* Add JSON versions of the remainder of the edge management stats

* Add new file to cmake

* Properly define management handler

* Only update the last updated timestamp after a successful data fetch

* Function and types definition cleanup

* Force correct type for python scripts mgmt port

* Implement initial JSON API for supernode

* Fix whitespace error

* Use helper function for rendering peers ip4 address

* Proxy the auth requirement back out to the http client, allowing normal http auth to be used

* Ensure that we do not leak the federation community

* Use the same rpc method name and output for both edge and supernode for peers/edges

* Allow n2nctl to show raw data returned without resorting to tricks

* Make n2nctl pretty printer understandable with an empty table

* Use the full name for supernodes RPC call

* Use same RPC method name (but some missing fields) for getting communities from both edge and supernode

* Add *_sup_broadcast stats to edge packet stats output

* Refacter the stats into a packetstats method for supernode RPC

* Even if I am not going to prettyprint the timestamps, at least make all the timestamps on the page the same unit

* Simplify the RPC handlers by flagging some as writable and checking that in the multiplexer

* Remove invalid edges data

* Avoid crash on bad data to verbose RPC

* Avoid showing bad or inconsistant protocol data in communities RPC

* Minor clarification on when --write is handled

* Make linter happy

* Fix changed method name in n2nhttpd

* Move mainloop stop flag into the n2n_edge_t structure, allowing access from management commands

* Implement edge RPC stop command

* Move mainloop stop flag into the n2n_sn_t structure, allowing access from management commands

* Implement supernode RPC stop command

* Allow multiple pages to be served from mini httpd

* Extract common script functions into a separate URL

* Handle an edge case in the python rpc class

With a proper tag-based demultiplexer, this case should be a nop,
but we are single-threaded and rely on the packet ordering in this
library.

* Add n2nhttpd support to query supernode using urls prefixed with /supernode/

* Handle missing values in javascript table print

* Add another less filtering javascript key/value renderer

* Add a supernode.html page to the n2nhttpd

* Address lint issue

* Mention the second html page on the Scripts doc

* Remove purgable column from supernode edges list - it looks like it is rarely going to be set

* Add a simple one-line example command at the top of the API documentation

* Acknowledge that this is not the most efficient protocol, but point out that it was not supposed to be

* Make it clear that the n2nctl script works for both edge and supernode

* Fight with inconsistant github runner results

* Turn off the /right/ coverage generator
2021-10-23 11:05:05 +05:45
Hamish Coleman
e6fcf1c55b
Cleanup and Documentation for JSON management API (#856)
* Reimplement JSON mgmt with clear separation of read/write actions

* Reduce boilerplate by using a table driven command definition for json mgmt commands

* Port tools to use new json api

* Add a basic authentication for json mgmt commands

* If a auth key is given, it must match

* Add auth key to management scripts

* Add a flag bitfield to clearly turn the tag param into a options list

* Allow simple pass-through of any command from n2nctl

* Convert the n2nctl to use an object oriented interface

* Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely

* Remove some repetition from the server

* Use the correct options to allow reuseaddr

* Dont generate a scary message on ctrl-c

* Convert n2nhttpd to use object based RPC

* Use the same longopt for both tools

* Pass any extra args through to the RPC

* Add some documentation for the scripts in the repository

* Spelling fix

* Add documentation for the JSON reply mangement API
2021-10-18 02:01:42 +05:45
Logan oos Even
966b6b9394
updated Scratchpad.md 2021-10-17 22:37:41 +05:45
Hamish Coleman
f5b730baed
added some cross-compiled binary outputs to the autobuild (#850)
* Add an example cross compile build

* Harmonise the naming to reflect full architecture and if it is a real package or not

* Add some more example cross compile targets

* Only one RPM package is created, so use the singular word

* Dont use build triplet for OS packages, use the OS prefered arch name

* Add some cross compiling documentation to the Building.md
2021-10-14 02:30:42 +05:45
Hamish Coleman
4438f1aa2a
added mingw test platform (#829)
* Provide a minimal reimplementation of our autoconf, to try windows builds

* Try building with windows

* Fix thinko in spelling

* Ensure shell script runs inside a shell

* Add a hack to aid include discovery

* Just keep adding tech debt...

* Assume that we will have slashes in some of the replacement strings and avoid that char with sed

* Restore one slash

* Hack around the tools makefile interdependancy bug

* A correct cflags include hack for each compile dir

* Ensure we link against winsock (note, even though this says 32bit, it should link the 64bit library ... I think)

* Bad link ordering if we dont use LDLIBS

* Remove unused make variable

* Remove makefile duplication using inheritance (this does mean you can no longer cd tools; make, but must do make tools)

* Add missing library for win32

* Show OS variable

* Make hack autoconf more robust for tests on non gitlab runners

* Remove no longer used substitutions from hack autoconf

* Add missing include path to tools under win32

* Build the win32 subdir when the compiler is Msys

* The different subdirs have different dependancies

* Ensure we can find the include files

* Fix library link ordering

* Ensure the tools dir can find the special win32 lib

* Deal with the differing basic type sizes on both linux/64bit and windows/64bit

* Document the steps to mimic the github windows/mingw build locally - to allow for simpler debugging

* Ensure branch name in instructions matches my test branch name

* Clarify the shell needed to build with mingw

* Since the makefile depends on knowing the OS, raise a fatal error if we cannot determine this

* Handling different compile environments is hard.

- Linux: sane and reasonable results for both uname -s (=Linux) and
  uname -o (=GNU/Linux)
- Windows/Mingw: insane results for uname -s
  (=MSYS_NT-$MAJOR.$MINOR-$BUILDNR) but sane results for uname -o (Msys)
- Macos: sane results for uname -s (=Darwin) but does not support
  uname -o at all

* Revamp the way that Mingw is detected

* Avoid attempting to generate gcovr report when running under windows

* Whoops, isolate the right step

* Fix spelling mistake

* win32/Makefile: Remove unused setting and add comment

* ensure that all win32 includes use the same expected path

* Allow simpler cross compilation by letting configure pass the CC and AR environment through

* Avoid multiple '_CRT_SECURE_NO_WARNINGS redefined' warnings

* Convert to a consolidated CONFIG_TARGET variable to select any different compile options

* Use the more generic printf defines to avoid warnings on mingw

* Update mingw build docs

* English better for reader happy make

* Address a number of mingw compiler warnings

* Fix Visual C compile

* Be sure to document some of the hacky nature of the mingw build
2021-10-06 00:52:15 +05:45
Logan oos Even
4aaa5e5522
updated Scratchpad.md 2021-09-27 19:55:21 +05:45
Logan oos Even
cc2b0b6b6b
added support for N2N_FEDERATION environment variable (#820) 2021-09-26 04:32:12 +05:45
Logan oos Even
a09d9ab698
updated Scratchpad.md 2021-09-25 16:10:38 +05:45
Logan oos Even
dfe15ad95d
added support for N2N_PASSWORD environment variable (#818) 2021-09-25 16:01:11 +05:45
Hacker
09fdfb0424
added support for COMMUNITY_NAME environment variable (#814)
Co-authored-by: codeneno <qsmy_qin@163.com>
Co-authored-by: Logan oos Even <Logan.00sEven@gmail.com>
2021-09-25 15:15:16 +05:45
Logan oos Even
92aa5ebee6
updated Scratchpad.md 2021-09-10 03:23:48 +05:45
Logan oos Even
4cba2b5582
typo 2021-09-08 01:11:19 +05:45
Logan oos Even
04d91f84d8
fixed optional SKIP_MULTICAST_PEERS_DISCOVERY macro (#770) 2021-09-08 00:56:05 +05:45
Logan oos Even
07cfa81d90
updated Scratchpad.md 2021-08-29 15:19:07 +05:45
Hamish Coleman
ecedcdcb58
Major documentation improvements (#752) 2021-08-19 20:02:53 +05:45
Logan oos Even
54ec208e5e
updated Federation.md 2021-08-08 18:01:38 +05:45
Logan oos Even
e09f3a4875
added federation support to user/password auth scheme (#741) 2021-07-31 22:17:07 +05:45
Logan oos Even
72d372cd04
updated scratchpad 2021-07-31 18:14:36 +05:45
Logan oos Even
9a7d05ebeb
updated Authentication doc 2021-07-21 14:17:35 +05:45
Logan oos Even
5a1c837f88
updated FAQ 2021-07-21 14:06:31 +05:45
Logan oos Even
045f6a7386
updated Faq.md 2021-07-20 22:38:23 +05:45
Logan oos Even
39aa50b4cd
added manual MAC/IP address spoofing protection override at supernode (-M) (#716) 2021-06-21 17:03:07 +05:45
Logan oos Even
a28327a0af
added user-password edge authentication (#673) 2021-06-04 03:11:22 +05:45