Commit Graph

245 Commits

Author SHA1 Message Date
Hamish Coleman
3ad14dc6a7 iwyu: missed deletions 2023-06-14 20:26:21 +01:00
Hamish Coleman
f67cc8f2a6 iwyu: random_numbers.h 2023-06-14 20:14:11 +01:00
Hamish Coleman
15ad901b41 iwym: pearson.h 2023-06-14 20:02:54 +01:00
Hamish Coleman
04ce678969 iwym: json.h 2023-06-14 19:52:10 +01:00
Hamish Coleman
4838dd8d75 iwym: header_encryption.h 2023-06-13 22:38:47 +01:00
Hamish Coleman
20df771af7 iwym: cc20.h 2023-06-13 21:45:11 +01:00
Hamish Coleman
ae3d54eed6 iwym: auth.h 2023-06-13 21:24:30 +01:00
Hamish Coleman
267b297150 Switch build date to use the last commit time
This reduces binary changes if the same commit version is re-built
2023-05-08 16:46:01 +01:00
Hamish Coleman
40c99fcdee
Merge pull request #1094 from hamishcoleman/autoconf
Autoconf Refactoring
2023-05-04 11:03:00 +08:00
Dylan Yee
e01daf4a85
support linux l2 bridge (#1044)
* support linux l2 bridge

* unified timeout name

* typo

* minor code style adaptions

* feature define

* feature define

* feature define

* feature define

* added hint to bridging to help output

* added hint to bridging

* drafted bridging documentation

* added bridging hint

---------

Co-authored-by: Logan oos Even <46396513+Logan007@users.noreply.github.com>
2023-05-04 11:00:38 +08:00
Hamish Coleman
1accc76ad3 Fix the building the win32 subdir.
It looks like the github windows build environment is not correctly
inheriting environment variables.  Which is strange, as that has not
been changed with the recent updates.  Include the vars directly as a
workaround (and plan to refactor later)
2023-04-30 20:40:50 -05:00
Hamish Coleman
3cef9949c2 Remove the non-maintained CMake build system.
It is difficult to maintain one build system, let alone two - especially
if one build system is known to be better maintained and the other build
system continues to be used - with bug and feature requests raised
against it.

The maintainers are aware that there are people using the CMake system,
but believe that the extra maintainability of only having one build
system, combined with the knowledge that the current CI system
demonstrates that all supported architectures are working with the
Makefiles make the use of these Makefiles a more sustainable direction.
2023-04-19 17:06:45 +08:00
Logan oos Even
2ee7dfdb98
added Windows support to n2n-route tool (#1023)
* added Windows support to n2n-route tool

* fixed includes

* more include fixes

* one more include addition

* more compile fixes

* wish I had a working Windows VM

* fixed some more Windows compile errors

* considered Windows-specific lib linkage

* promised to never code OS specific tools again

* removed invisible invalid characters

* where to get if_idx from

* retrieving interface index for route

* bracket...

* one more bracket...

* added optional gateway parameter to user-defined routes

* clarification

* clarification

* Windows needs special network init

* adapted waiting-for-key
2023-02-03 17:01:57 +01:00
Logan oos Even
cf23457d3b
added sock type field to data structure (#1031) 2022-06-28 19:09:04 +02:00
Logan oos Even
053ab381b3
fixed endianess issue in federated sn socket forwarding (#1030) 2022-06-28 16:53:12 +02:00
Logan oos Even
95cfbca5a9
more sockaddr generalization (#1028) 2022-06-27 22:21:03 +02:00
Logan oos Even
439dfc6886
generalized sockaddr usage in edge (#1026) 2022-06-26 17:17:22 +02:00
Logan oos Even
ffb4e31b8a
encapsulated several fillings of n2n_sock_t in single function (#1025)
* encapsulated several fillings of n2n_sock_t in single function

* Windows specifics...

* stuck to the project's style code
2022-06-26 00:07:29 +02:00
Logan oos Even
7927bd7294
added optional choice of bind address (-p) to supernode (#1016) 2022-06-16 12:00:48 +02:00
Logan oos Even
51a3a71f95
introduced n2n-portfwd tool (#1008)
* moved dev to version 3.1.1

* introduced n2n-portfwd tool

* moved port-forwarding source to tool

* adjusted tiny things, seemingly works with ./configure's  --enable-miniupnp  and  --enable-natpmp

* adjusted for Windows

* wished for better typing skills

* applied some finishing touch

* typo
2022-06-05 18:05:40 +02:00
Logan oos Even
99b6b6b66d
added n2n-route tool (#982)
* moved dev to version 3.1.1

* laid ground for n2n-route tool

* adapted code style

* overhauled n2n-route's tool program logic and removed route code from edge

* added missing initialization of federation's purgeable field

* lifted un/purgeable confusion

* added warning about removed -n cli option

* realized that Windows does not offer inet_aton()

* removed -n option documentation from edge's man page

* slightly simplified n2n-route program logic

* applied more logic changes to n2n-route tool

* added 'info' read command to edge's management port

* corrected indention

* added Linux route control to n2n-route tool

* temporarily restricted n2n-route tool to Linux only

* We must be over the routing!

* pulled default gateway change detection into main loop to cover mobile use, and added devstr_t type

* corrected use of new UNPURGEABLE - so far gone unnoticed

* addresses possiible address issue

* i broke it

* reverted bad ideas

* added command line options, help text, and prevented vpn gateway being used for supernode/peer traffic routes

* added option to manually provide default gateway, also verbosity options

* getting there

* added option to limit networks to be routed (-n), adapted documentation

* fine-tuned minor things
2022-05-24 20:38:51 +02:00
Luca Deri
aecb97042e Added missing dependency #989 2022-05-06 22:29:49 +02:00
Hamish Coleman
3fa0fa1bbe
Merge pull request #983 from hamishcoleman/json_pubsub
Refactoring management code to increase sharing
2022-04-30 18:21:36 +01:00
Hamish Coleman
009311d016
Provide better details when sendto_fd fails (#981)
* Refactor traceEvent to allow variable log levels

* Refactor sendto failed for less repetition

* Refactor sendto_fd to make control flow clearer, also reduce repetition

* Tie size to the actual object not its class

* Show destination that had issues when sendto_fd fails

* Fix MSVC compile - it doesnt support the same variadic macros as gcc
2022-04-25 19:13:14 +02:00
Hamish Coleman
16f7f4dc59 Fiddle with windows headers 2022-04-17 00:37:02 +01:00
Hamish Coleman
9728a36a56 Increase includes reusability and consistency 2022-04-17 00:25:04 +01:00
Logan oos Even
e686b9592c
(C) update (#965) 2022-03-12 10:22:42 +01:00
Logan oos Even
f59c6900a1
moved compression code into transforms (#961) 2022-02-21 10:09:06 +01:00
Hamish Coleman
860b27d97c Add event publishing for adding and removing P2P peers 2022-02-05 20:06:31 +00:00
Hamish Coleman
a40ed15d93 Add pubsub system to edge, with one example TEST channel 2022-02-05 20:06:31 +00:00
Hamish Coleman
3fd1dc5fa8 keep private enum definition local to the management code 2022-02-05 20:06:31 +00:00
Hamish Coleman
ace3dbc44f Move all supernode management handling into one source file 2022-02-05 20:06:31 +00:00
Hamish Coleman
a8e2704b41 Move all management packet handling for edge into the one source file 2022-02-05 20:06:31 +00:00
Hamish Coleman
4f568b03c1
Autoconf simplification (#927)
* Convert zstd feature to default disable

* All autoconf test use one standard template

To simplify the testing, cross-compilation and repeatable build process,
no configure options are automatically probed for - they all default to
off and are all using the same template.

The --with-x options should be deprecated and replaced with --enable-x
because there is no syntax checking for --with options in autoconf.

There are still some differences between the config options, but this
should provide a starting point.

* Remove unused code from the autoconf

* Remove warnings from default build

* Avoid calling port mapping functions if none are enabled

* Start with all builds in neutral config

* Add more missing code guards

* Adjust code guard location to placate cmake
2022-01-10 19:58:15 +00:00
Logan oos Even
b657881a92
differentiated miniupnp includes (#912) 2021-12-24 10:32:22 +01:00
Logan oos Even
e6e8cb038a
added port forwarding (upnp and natpmp) (#905)
* UPnP port redirection is supported.

* compile fixes

* compile fix

* optimize reconnection code

* prepared upnp threadification to counter main loop stall at supernode change

* NAT-PMP port forwarding support, temporarily merge codes to resolve conflicts.

* make compile fix

* prepared threadification in more detail

* adopted threadification to new file setup

* cleaned up

* renamed functions and data structures

* fixes

* differentiated between miniupnp and natpmp and added corresponding lib support to makefile

* name

* commented unused header includes

* comments

* license

* fixes

* fixes

* fixes

* NAT-PMP is already available.

* added CLI parameter to disable port forwarding if required

* preliminary made use of multithreading

* adjusted log level

* added man page documentation

* def'ed conf

* made pmpnat adjustments

Co-authored-by: fengdaolong <fengdaolong@gmail.com>
2021-12-23 12:27:55 +01:00
Logan oos Even
89571292f4
allowed supernode MAC address to be set manually (-m) (#886) 2021-11-01 17:19:46 +05:45
Logan oos Even
f4e7e513a8
added MAC address based supernode selection (#885) 2021-11-01 16:35:05 +05:45
Logan oos Even
3b187b4ac8
updated JSON API password handling '--management-password <pw>' (#869) 2021-10-24 15:22:36 +05:45
Logan oos Even
c61b62ab1b
choose supernode selection strategy at run-time '--select-rtt' (#864) 2021-10-24 12:53:25 +05:45
Hamish Coleman
80b33cd1a9
Basic C Code lint checker and shell checker (#859)
* Factor build packages out into a more maintainable list

* Create a location for scripts to live

* Provide a make target to return the source dir as close as reasonable to the original distributed state

* Add a code lint step, checking the coding style

* Change test harness as recommended by shellcheck

* Ensure we actually have the linter tool installed

* Use the correct directory for cmake to run the tests

* Adjust for the older uncrustify in the current github ubuntu-latest

* Make one file pass the linter

* Integrate the lint with the existing test workflow

* Add files with minimal changes needed to the linter

* Add more files with minimal changes needed to the linter

* Dont build binaries if we fail the lint test

* Update the phony targets with the lint steps

* Ensure the flake8 package is installed in the new lint workflow job

* Use the makefile to drive the packages needed to install for linting

* No need to add dependancies on lint, just rely on the workflow status to show failure

* Update the scripts dir README to reflect current assumptions

* Rename and briefly document the indent.sh script

* Fix the ignore to ignore the right Makefile

* Rename the test_harness script to make it clear it is a shell script

* Provide a master lint make target and add a shell script lint tool

* Elminate stray tabs

* Drop include/auth.h from linter - there are inconsistant results with function definitions when using the current uncrustify rules
2021-10-24 01:21:18 +05:45
Hamish Coleman
ae502d9181
JSON Reply Management API - feature parity with old management interfaces (#861)
* Ensure that recent code additions pass the linter

* Include some of the more obviously correct lint fixes to edge_utils.c

* Refactor edge JSON api into its own source file

* Use shorter names for static management functions

* Implement a JSON RPC way of managing the verbosity

* Tidy up help display in n2nctl script

* Make note of issue with implementing the stop command

* Implement a JSON RPC call to fetch current community

* Make n2nhttpd time value be more self-contained

* Make n2nhttpd order more closely match the existing management stats output

* Wire up status page to the verbosity setting

* Add JSON versions of the remainder of the edge management stats

* Add new file to cmake

* Properly define management handler

* Only update the last updated timestamp after a successful data fetch

* Function and types definition cleanup

* Force correct type for python scripts mgmt port

* Implement initial JSON API for supernode

* Fix whitespace error

* Use helper function for rendering peers ip4 address

* Proxy the auth requirement back out to the http client, allowing normal http auth to be used

* Ensure that we do not leak the federation community

* Use the same rpc method name and output for both edge and supernode for peers/edges

* Allow n2nctl to show raw data returned without resorting to tricks

* Make n2nctl pretty printer understandable with an empty table

* Use the full name for supernodes RPC call

* Use same RPC method name (but some missing fields) for getting communities from both edge and supernode

* Add *_sup_broadcast stats to edge packet stats output

* Refacter the stats into a packetstats method for supernode RPC

* Even if I am not going to prettyprint the timestamps, at least make all the timestamps on the page the same unit

* Simplify the RPC handlers by flagging some as writable and checking that in the multiplexer

* Remove invalid edges data

* Avoid crash on bad data to verbose RPC

* Avoid showing bad or inconsistant protocol data in communities RPC

* Minor clarification on when --write is handled

* Make linter happy

* Fix changed method name in n2nhttpd

* Move mainloop stop flag into the n2n_edge_t structure, allowing access from management commands

* Implement edge RPC stop command

* Move mainloop stop flag into the n2n_sn_t structure, allowing access from management commands

* Implement supernode RPC stop command

* Allow multiple pages to be served from mini httpd

* Extract common script functions into a separate URL

* Handle an edge case in the python rpc class

With a proper tag-based demultiplexer, this case should be a nop,
but we are single-threaded and rely on the packet ordering in this
library.

* Add n2nhttpd support to query supernode using urls prefixed with /supernode/

* Handle missing values in javascript table print

* Add another less filtering javascript key/value renderer

* Add a supernode.html page to the n2nhttpd

* Address lint issue

* Mention the second html page on the Scripts doc

* Remove purgable column from supernode edges list - it looks like it is rarely going to be set

* Add a simple one-line example command at the top of the API documentation

* Acknowledge that this is not the most efficient protocol, but point out that it was not supposed to be

* Make it clear that the n2nctl script works for both edge and supernode

* Fight with inconsistant github runner results

* Turn off the /right/ coverage generator
2021-10-23 11:05:05 +05:45
Hamish Coleman
e6fcf1c55b
Cleanup and Documentation for JSON management API (#856)
* Reimplement JSON mgmt with clear separation of read/write actions

* Reduce boilerplate by using a table driven command definition for json mgmt commands

* Port tools to use new json api

* Add a basic authentication for json mgmt commands

* If a auth key is given, it must match

* Add auth key to management scripts

* Add a flag bitfield to clearly turn the tag param into a options list

* Allow simple pass-through of any command from n2nctl

* Convert the n2nctl to use an object oriented interface

* Handle sigpipe in the n2nhttpd - this happens if the remote client disconnects unexpectely

* Remove some repetition from the server

* Use the correct options to allow reuseaddr

* Dont generate a scary message on ctrl-c

* Convert n2nhttpd to use object based RPC

* Use the same longopt for both tools

* Pass any extra args through to the RPC

* Add some documentation for the scripts in the repository

* Spelling fix

* Add documentation for the JSON reply mangement API
2021-10-18 02:01:42 +05:45
Hamish Coleman
4438f1aa2a
added mingw test platform (#829)
* Provide a minimal reimplementation of our autoconf, to try windows builds

* Try building with windows

* Fix thinko in spelling

* Ensure shell script runs inside a shell

* Add a hack to aid include discovery

* Just keep adding tech debt...

* Assume that we will have slashes in some of the replacement strings and avoid that char with sed

* Restore one slash

* Hack around the tools makefile interdependancy bug

* A correct cflags include hack for each compile dir

* Ensure we link against winsock (note, even though this says 32bit, it should link the 64bit library ... I think)

* Bad link ordering if we dont use LDLIBS

* Remove unused make variable

* Remove makefile duplication using inheritance (this does mean you can no longer cd tools; make, but must do make tools)

* Add missing library for win32

* Show OS variable

* Make hack autoconf more robust for tests on non gitlab runners

* Remove no longer used substitutions from hack autoconf

* Add missing include path to tools under win32

* Build the win32 subdir when the compiler is Msys

* The different subdirs have different dependancies

* Ensure we can find the include files

* Fix library link ordering

* Ensure the tools dir can find the special win32 lib

* Deal with the differing basic type sizes on both linux/64bit and windows/64bit

* Document the steps to mimic the github windows/mingw build locally - to allow for simpler debugging

* Ensure branch name in instructions matches my test branch name

* Clarify the shell needed to build with mingw

* Since the makefile depends on knowing the OS, raise a fatal error if we cannot determine this

* Handling different compile environments is hard.

- Linux: sane and reasonable results for both uname -s (=Linux) and
  uname -o (=GNU/Linux)
- Windows/Mingw: insane results for uname -s
  (=MSYS_NT-$MAJOR.$MINOR-$BUILDNR) but sane results for uname -o (Msys)
- Macos: sane results for uname -s (=Darwin) but does not support
  uname -o at all

* Revamp the way that Mingw is detected

* Avoid attempting to generate gcovr report when running under windows

* Whoops, isolate the right step

* Fix spelling mistake

* win32/Makefile: Remove unused setting and add comment

* ensure that all win32 includes use the same expected path

* Allow simpler cross compilation by letting configure pass the CC and AR environment through

* Avoid multiple '_CRT_SECURE_NO_WARNINGS redefined' warnings

* Convert to a consolidated CONFIG_TARGET variable to select any different compile options

* Use the more generic printf defines to avoid warnings on mingw

* Update mingw build docs

* English better for reader happy make

* Address a number of mingw compiler warnings

* Fix Visual C compile

* Be sure to document some of the hacky nature of the mingw build
2021-10-06 00:52:15 +05:45
Logan oos Even
db5adc2038
fine-tuned REGISTER cookie handling (#837) 2021-10-03 18:19:29 +05:45
Logan oos Even
bc65a7f59e
added forwarded REGISTER cookie type (#836) 2021-10-03 05:24:33 +05:45
Logan oos Even
ebb357245a
added a flag field to QUERY_PEER (#835) 2021-10-02 23:17:45 +05:45
Logan oos Even
2d34545304
compile fix (#834) 2021-09-30 12:39:31 +05:45
Logan oos Even
16d248d921
fixed MinGW compile (#830) 2021-09-30 02:20:48 +05:45
Hamish Coleman
b735ad6b9e
added test framework and code coverage reporting (#797)
* Add a simple test framework

* Add a code coverage report example oneliner

* Move the coverage report into a separate directory

* Add a github action to run tests and publish a branch with the coverage report

* Fix: Missing job separator

* Fix: remember to actually run configure

* Fix: Gotta autogen before I configure

* Dont try to upload coverage report unless this is a push

* Clearly show the git ref tested in the coverage report

* Add a test for the various transforms

* Add tests for the elliptic curve and pearson hash

* Ensure we ignore new generated output

* Remove unneeded boilerplate from the compression tests

* Add an example of a test of the encoded wire packets

* Ensure that correctly testable data is output even when zstd is not compiled

* Factor test runner out into its own script and attempt to add it to the cmake file

* Tell cmake about a new object file

* Stop trying to make Cmake work...

* Stop trying to make cmake work, round 2

* In the middle of a thousand lines of cmake output was one important one - windows could not find assert() - try again

* Try again to plumb the tests into cmake

* Add missing library to our superset install line

* Fix build error when libcap-dev is installed

* Switch to using artifact uploads instead of pages to store/show the coverage report

* Fix artifact upload yaml

* Upload coverage report to codecov

* Fix codecov - clearly it doesnt do a recursive search for coverage files

* Fix codecov - my hopeful use of a list of directories didnt work

* Fix codecov - unfortunately, it doesnt just consume the coverage data and needs us to generate the gcov output

* Fix codecov - nope, it still doesnt recursively search

* Fix codecov - it really helps if I run the gcov data generator

* Add a simple matrix build

* Fix older ubuntu versions of gcovr that do not support the '--html-title' option

* Ensure we use gcover options that are identical on older ubuntu

* Improve coverage generation and required build packages
2021-09-27 15:26:06 +05:45