ETHERPAD

< DRAFT+ >

The self-hosted realtime, collaborative pads. Also, powering this site via 🔗E2H.

Revisions:
    20190709 stub
    20191001 touch
    20200510 touch
    20200906 pre-pub, push to draft
    20230420 installing on localhost (with some problems...)

Ties to:
    🔗E2H
    ^XXX🔗mima22-api

Table of Contents

1	*** BASIC RESOURCES
2	*** INSTALLATION
2.1	(conditions)
2.2	2023/04 take
2.3	settings.json
2.4	Apache VirtualHost configuration (proxypass)
2.4.1	(on localhost) localhost/pad/### → localhost:9001/###
2.4.2	(on server) pad.totalism.org/### → localhost:9001/###
2.5	(20220319) Updating etherpad & node & npm ... from very old versions
2.6	[m!!!] OWN DOCS
2.7	EXISTING DOCS
3	*** BACKUP + RESTORE
4	*** BUGS
4.1	* picking (B/I/U/S) styles does not work:
4.2	* default new UI contrast not optimal:
4.3	* WEIRD CURSOR BEHAVIOURS on wrapped line:
4.4	* missing: stable link to latest version of documentation:
4.5	* 100% cpu, then FAILURE (via timeouts) on moving bigger pads:
4.6	* you have space indented lines:
4.7	* you're on a tabulator indented line & line above it is empty:
4.8	* loading delay in timeslider view, without any indication:
4.9	* connection failures (with lost content) happen all the time:
4.10	* try copying in something from gnome terminal, as "Copy as HTML" (with right click):
4.11	* copying in a tab-space, it default to 8 characters:
4.12	* ... also tabulation should be (optionally, at least) spaces based:
4.13	* listallpads API should include time (and maybe revision no):
4.14	* very big pads fail to export .etherpad
4.15	* fail to "import" very big .etherpad files (30MB)
4.16	* timeslider date format is weird:
4.17	* (2020/11) On pad disconnect (sleep→restore, momentary connection loss, ...), this scary message awaits you in a red tab
4.18	* you have to first make an edit to be able to see the "import" button
4.19	* etherpad (and also html export) fucks up on specific characters, ends URL/link
4.20	* ... does same for " ' ", see
4.21	* there is no way to paste long strings without having them broken up at ??? character
4.22	* inconsistent url interpretation between editor and html export:
4.23	* you copy-paste a line that has some empty lines after it:
4.24	* somebody is writing at the very end of the document. no empty line after it. you're unable to be adding something later:
4.25	* undercores in pad names, are dropped from tab title
4.26	* timeslider export/settings buttons don't pop up on top
4.27	* URLs ending in ")" bracket are not put together correctly
4.28	* padnames convert "_" in tab titles to spaces
4.29	* splitting on long lines (seems at 2000 characters)
4.30	cursed copy sequence
4.31	icon seems missing?
4.32	no-message pad crash on big copy
4.33	[k?] "bad requst" 400 (in console)
4.34	[!!*] bug: rename of bigger pads fails (timeouts)
4.35	* pgup is super fast. pgdn is super slow.
4.36	* sync error on trying to undo clear authorship
4.37	* copy-pasting this text (from chatgpt) throws js error, breaks editor, doesn't persist
4.38	[!] BUG/CUTS— (Artefact copy-pasting this in)
4.39	[r!] MAINTAIN— (PNPM upgrade)
5	*** FEATURE REQUESTS
5.1	* merge two pads & preserve history!:
5.2	* timeslider BLAME:
5.3	* timeslider SEARCH:
5.4	* "suggestions mode":
5.5	* "limited edit mode":
6	*** EFFECTIVE COLLABORATION ON PADS
7	*** TIPS
7.1	Running localhost Etherpad offline? disable "installDeps.sh" !
7.2	500 internal error for proxypass ... this did it!
7.3	Troubleshooting: Instance won't start after upgrade
7.4	Upgrade Etherpad version
8	*** PAD
8.1	Etherpad API / alternative clients / editors
8.2	Etherpad alternatives: other collab online editors / libraries
8.3	[!!**o] EXPLORE— CHECK THESE PLUGINS
8.4	[!!!**] TOMERGE— My resources from "###linux"
8.5	[!!**] PUBLISH— Publish this site, by linking it to...
8.6	[!!] MISSING— Instance "sanity check" resource
8.7	[!!!**] CONTRIBUTE/PUBLISH— Finish and report the bugs & feature requests
8.8	[~] TRICK— Check out "/health" endpoint
8.9	[r!!*] (etherpad maintenance)
8.10	[!!s] Refresh Etherpad: Just start again :)
8.11	[!!**tt] Etherpad seems full of vulnerabilities, is behind on updating packages, running old npm?
8.12	[!!!**] BUG— instability ~2024
8.13	————— 20240227 version refresh (with npm/node version bumps!)
8.14	[z!] (ERROR, was shown on client-side)
8.15	[!!!**] NEW PROBLEMS ... CONSTANT DROPS AGAIN?!
8.16	[!!**$] DOCS— (Generally, there should be a "what's new" or "upgrade guide")
8.17	[a!!$] INSTALL— Setup Etherpad logging to file
8.18	[a!!!$] BUG!!/TOEMAIL/TROUBLES— Upgrading ~2024/2, pad loading fails because websockets line in settings.json
8.19	[a!] BUG?/TROUBLES— (UPDATE PROBLEMS 20240228@luna and 20240414@local)
8.20	[!*t] BUG?/TROUBLES?— (More instability, not sure if temporary, or will persist and pester?)
8.21	[rr!!*efo] ADMIN/MAINTAIN— Review changelogs! (Etherpad 2.0+)
8.22	[!!*oo] PLUGIN— Audio input via ep_padgpt
8.23	[!!*o] PADLAND— Fix the nicer padland links
8.24	[!!*ot] ACL/— Have pass
8.25	[!!!**t] BUG/INSTALL/UNSTABLE— Frequent disconnect problems
8.26	[!!**et] BUG— Scrub the site of "npm install"
8.27	[!!**eef] DEV/SETTINGS— Specify date format for "play" mode :)

¶*** BASIC RESOURCES

* https://etherpad.org/

* Etherpad What's new:
https://github.com/ether/etherpad-lite/blob/develop/CHANGELOG.md

* https://discord.com/channels/741309013593030667/741309013593030670
Discord channel

* https://github.com/ether/etherpad-lite/wiki
Wiki documentation

* https://docs.etherpad.org :
new documentation page

* https://etherpad.org/doc/latest
API documentation

* https://blog.etherpad.org/

* http://etherpad.idea.informer.com/
Public feature requests...

* ### [!!]
Mailing list

¶*** INSTALLATION

¶(conditions)

A decent production-level installation instruction for etherpad seems missing.

Must include:
    * has mysql with utfmb4
    * has ssl (let's encrypt)
    * has restarter
    * [...]

¶2023/04 take

first setup mysql
https://www.digitalocean.com/community/tutorials/how-to-install-mysql-on-ubuntu-20-04

then etherpad
via https://github.com/ether/etherpad-lite/wiki/How-to-use-Etherpad-Lite-with-MySQL

setup the service
https://github.com/ether/etherpad-lite/wiki/How-to-deploy-Etherpad-Lite-as-a-service
→→(follow "debian 8 (systemd"), BUT:
    * don't use /srv
    * make it a login shell (so you can later setup nvm):
        ####if you forgot
        $ usermod -s /bin/bash etherpad-lite
        $ passwd ####
    * ExecStart=/SW/etherpad-lite/run-with-nvm.sh
———
#!/bin/bash
echo "——————————————GO"
export NVM_DIR="/SW/nvm"
[ -s "$NVM_DIR/nvm.sh" ] && . "$NVM_DIR/nvm.sh"
/SW/etherpad-lite/bin/run.sh
echo "——————————————FIN"
—————

...change settings.json ...
(see ↓↓)

...problems...
* had to run "installDeps", but doesn't seem to be great:
    (prob Cuba situation)

* upgrading node via the tar.xz way:
    download straigth from https://nodejs.org/en
    then https://stackoverflow.com/questions/63312642/how-to-install-node-tar-xz-file-in-linux

* when running, getting some errors:
    [2023-04-20 02:04:52.004] [ERROR] ueberDB - Fatal MySQL error: Error: ER_NOT_SUPPORTED_AUTH_MODE: Client does not support authentication protocol requested by server; consider upgrading MySQL client
--------------versions? (after upgrade)

dcht00@chetvertyy:~$ node -v ; npm -v
v18.16.0
8.8.0

---------------

* tried uninstall mysql-client , installing mariadb-client → nope

* installed nvm :
    via https://tecadmin.net/how-to-install-nvm-on-ubuntu-20-04/
    $ curl https://raw.githubusercontent.com/creationix/nvm/master/install.sh | bash
    ----------------
    UPDATE:
        * moved it to /SW/nvm, yeah
        * not ideal apparently

$ nvm --lts
#versions now
v18.16.0
9.5.1
------------

* removed a bunch of package.lock files (already before), now problems with this
$ cd bin ; npm install ....... # ~10min, "up to date" after repeat
$ ./installDeps ... # looks good

-------------- trying to run
# had some shit with mysql, was not running:
    # check with netcat
# see it's "dead" (prob for the client reinstall before)

root@chetvertyy:/home/dcht00# systemctl restart mysql
↓
"Failed to restart mysql.service: Unit mysql.service is masked."
# wtf?
↓
$ systemctl unmask mysql.service
$ systemctl restart mysql.service

------------ ok it runs ... but not with systemctl, because user "etherpad-lite" has no npm/nvm... ----------
$ sudo su -l etherpad-lite -s /bin/bash

-------------- ok, of course this nvm is giving me problems, I follow this ------------
https://github.com/nvm-sh/nvm/issues/1533
and kind of get it to work?

------------- and it runs!

-------------- but next day ... it gets stuck at the installDeps package (maybe Cuba factor, definitely offline factor)
↓
just throw this out of run.sh:
"""
# Prepare the environment
###DAVID WTF ################################# DISABLE THIS FOR LESS PROBLEMS
#src/bin/installDeps.sh "$@" || exit 1
"""
and we good.

--------- also, fix all the a2emod

"""
You will need to activate (maybe after having installed) the mod_proxy, mod_proxy_http, mod_headers, proxy_wstunnel, mod_deflate and mod_rewrite Apache modules.
"""
5250 27/04/23 07:14:33 sudo a2enmod proxy
5251 27/04/23 07:14:37 sudo a2enmod proxy_http
5252 27/04/23 07:14:41 sudo a2enmod headers
5256 27/04/23 07:14:55 sudo a2enmod proxy_wstunnel
5257 27/04/23 07:15:01 sudo a2enmod deflate
5258 27/04/23 07:15:05 sudo a2enmod rewrite
5248 27/04/23 07:12:51 sudo systemctl restart apache2.service

—————————— happy end

¶settings.json

* mysql (obviously):
    port → /var/run/mysqld/mysqld.sock

"skinName": "no-skin",
    "showChat":         false,
    "useMonospaceFont": true,

### (CURRENTLY TESTING ON TOTALISM.ORG, NOT SURE WHAT IT DOES)
"trustProxy": true,

### [!!] CONSIDER TO CHANGE THIS, FOR WSS PROBLEMS
"socketTransportProtocols" : ["xhr-polling", "jsonp-polling", "htmlfile"]

### MOST IMPORTANT ........ default 10kb, now 10MB
    "maxHttpBufferSize": 11111111

###
"importExportRateLimiting": {
    // duration of the rate limit window (milliseconds)
    //90000→1000
    "windowMs": 90000,

    // maximum number of requests per IP to allow during the rate limit wi
    //###10→100
    "max": 10
},

###
"commitRateLimiting": {
    // duration of the rate limit window (seconds)
    "duration": 1,

    // maximum number of changes per IP to allow during the rate limit win
    //###10→1000
    "points": 1000
},

### LOG
"logconfig" :
    { "appenders": [
        { "type": "console"
        //, "category": "access"// only logs pad access
        }

      , { "type": "file"
      , "filename": "etherpad_totalism.log"
      , "maxLogSize": 66666666
      , "backups": 2 // how many log files there're gonna be at max
      //, "category": "test" // only log a specific category
        }
    ###(etc)

--------------------- OKAY, done

¶Apache VirtualHost configuration (proxypass)

¶    (on localhost) localhost/pad/### → localhost:9001/###

for localhost with ssl:
    place this in /etc/sites-available/default-ssl.conf
    (within the default VirtualHost block)

----------------------------------------------------------------------------------------

RewriteEngine On
RewriteRule ^/pad$ /pad/ [R]
ProxyPass /pad/ http://localhost:9001/
ProxyPassReverse /pad/ http://localhost:9001/
ProxyPreserveHost on

    <Location /pad/socket.io>
        # This is needed to handle the websocket transport through the proxy, since
        # etherpad does not use a specific sub-folder, such as /ws/ to handle this kind of traffic.
        # Taken from https://github.com/ether/etherpad-lite/issues/2318#issuecomment-63548542
        # Thanks to beaugunderson for the semantics

        RewriteEngine On
        RewriteCond %{QUERY_STRING} transport=websocket    [NC]
        RewriteRule /(.*) ws://localhost:9001/socket.io/$1 [P,L]
        ProxyPass http://localhost:9001/socket.io retry=0 timeout=30
        ProxyPassReverse http://localhost:9001/socket.io

    </Location>

----------------------------------------------------------------------------------------

¶    (on server) pad.totalism.org/### → localhost:9001/###

----------------------------------------------------------------------------------------

<VirtualHost *:80>
    ServerName pad.totalism.org
    Redirect / https://pad.totalism.org/
</VirtualHost>

<VirtualHost *:443>
    ServerName pad.totalism.org

    # SSL configuration
    SSLEngine on
    SSLCertificateFile      /etc/letsencrypt/live/totalism.org/cert.pem
    SSLCertificateKeyFile   /etc/letsencrypt/live/totalism.org/privkey.pem
    SSLCertificateChainFile /etc/letsencrypt/live/totalism.org/fullchain.pem

    # Proxy
    ProxyVia On
    ProxyRequests Off
    ProxyPreserveHost on

~~############### TRYING 20220506~~
#
# ~~https://stackoverflow.com/questions/38400168/cors-access-control-allow-origin-on-apache-proxy-issue~~
#
~~##→→DISABLING (still won't work to read iframes)~~
#
~~#Header always set Access-Control-Allow-Origin "*"~~
~~#Header always set Access-Control-Allow-Headers "*"~~
~~#Header always set Access-Control-Allow-Methods "*"~~
~~##Header add Access-Control-Allow-Headers "origin, x-requested-with, content-type"~~
~~##Header add Access-Control-Allow-Methods "PUT, GET, POST, DELETE, OPTIONS"~~
#
~~###########################~~

    <Location />
        ProxyPass http://localhost:9001/ retry=0 timeout=30
        ProxyPassReverse http://localhost:9001/
    </Location>
    <Location /socket.io>
        # This is needed to handle the websocket transport through the proxy, since
        # etherpad does not use a specific sub-folder, such as /ws/ to handle this kind of traffic.
        # Taken from https://github.com/ether/etherpad-lite/issues/2318#issuecomment-63548542
        # Thanks to beaugunderson for the semantics

        RewriteEngine On
        RewriteCond %{QUERY_STRING} transport=websocket    [NC]
        RewriteRule /(.*) ws://localhost:9001/socket.io/$1 [P,L]
        ProxyPass http://localhost:9001/socket.io retry=0 timeout=30
        ProxyPassReverse http://localhost:9001/socket.io

       ProxyPass http://localhost:9001/socket.io retry=0 timeout=30
       ProxyPassReverse http://localhost:9001/socket.io
    </Location>

    <Proxy *>
        Options FollowSymLinks MultiViews
        AllowOverride All
        Order allow,deny
        allow from all
    </Proxy>

</VirtualHost>

----------------------------------------------------------------------------------------

¶(20220319) Updating etherpad & node & npm ... from very old versions

__________ NODE
aka "upgrade node"
via #https://stackoverflow.com/questions/63312642/how-to-install-node-tar-xz-file-in-linux
""""
node --version # too old! check "requirements" at ### on etherpad.org

### stop all instances
### [!] (and make sure they don't restart, in "root" crontab)
root@luna:/home/dcht00# systemctl stop etherpad-lite-padland
root@luna:/home/dcht00# systemctl stop etherpad-lite
root@luna:/home/dcht00# systemctl stop etherpad-lite-tlm

### [!] get tar from https://nodejs.org , the linux x64 version, then ...
sudo tar -xvf node-v*-linux-x64.tar.xz
sudo cp -r node-v*linux-x64/{bin,include,lib,share} /usr/

### okay!
node --version
"""

__________ NPM
(...actually and maybe also node?)
aka "upgrade NPM"

"""
npm -v
sudo su
npm cache clean -f
npm install -g n #node
npm install -g npm #6.13.4→8.5.5
npm -v
"""

___________ EXTRA?

#output offered patch so had to go explicit
$ npm install -g npm@8.5.5

#now """found 0 vulnerabilities"""

### (UPDATE: but AFAIR, this was a bad idea, as it was TOO NEW for etherpad...)

___________ (now, try to run etherpad)

* super long npm loading
* misses "express-session"

* looks like running! →1.9.0

############## problems

* "too many requests"

↓↓↓

"""
/*
   * From Etherpad 1.8.3 onwards, import and export of pads is always rate
   * limited.
   *
   * The default is to allow at most 10 requests per IP in a 90 seconds window.
   * After that the import/export request is rejected.
   *
   * See https://github.com/nfriedly/express-rate-limit for more options
   ### CHANGED
   */
"importExportRateLimiting": {
    // duration of the rate limit window (milliseconds)
    "windowMs": 1000,

    // maximum number of requests per IP to allow during the rate limit window
    "max": 100
},

/*
   * From Etherpad 1.8.5 onwards, when Etherpad is in production mode commits from individual users are rate limited
   *
   * The default is to allow at most 10 changes per IP in a 1 second window.
   * After that the change is rejected.
   *
   * See https://github.com/animir/node-rate-limiter-flexible/wiki/Overall-example#websocket-single-connection-prevent-flooding for more options
### CHANGED
   */
"commitRateLimiting": {
    // duration of the rate limit window (seconds)
    "duration": 1,

    // maximum number of changes per IP to allow during the rate limit window ... ###10→1000
    "points": 1000
},

"""

############## works perfectly

¶[m!!!] OWN DOCS

See ^dcht00🔗##### !

¶EXISTING DOCS

official (basic):
    * https://github.com/ether/etherpad-lite/wiki/How-to-use-Etherpad-Lite-with-MySQL :
        (BAD, SHOULD WORK TO CHANGE !!!)

official (extra):
    * https://github.com/ether/etherpad-lite/wiki/How-to-put-Etherpad-Lite-behind-a-reverse-Proxy :
        using this & it is fine (2020/10)

new / tocheck:
    ___________________ look great!
    * https://lab.uberspace.de/guide_etherpad.html !!!
    * https://www.howtoforge.com/tutorial/ubuntu-etherpad-editor-server-installation/ :
        "enable" systemd services
    * https://wiki.evolix.org/HowtoEtherpad
    ___________________ NO
    * https://vanad.is/how-to-host-etherpad.html (no utf8mb4 mysql certainty)
    ___________________ ???
    * https://bytesandbones.wordpress.com/2019/06/17/install-etherpad-on-debian/
    * https://stackoverflow.com/questions/46075120/how-to-debug-invalid-data-in-mysql-utf8mb4-column-in-etherpad-lite-database?rq=1
    * https://www.openattic.org/posts/how-to-install-etherpad-on-debian-jessie/
    * [...]

¶*** BACKUP + RESTORE

(EXISTING DOCS) ... meh:
* https://github.com/ether/etherpad-lite/wiki/Backing-up-and-Restoring-Etherpad-Lite-Pads
* [...]

###

¶*** BUGS

________________ 20200510

¶* picking (B/I/U/S) styles does not work:
    1) at start of line, they don't even appear selected on buttons
    2) in other positions, they appear selected, but deselect on next keypress
    so, they only for selecting and styling existing text)

¶* default new UI contrast not optimal:
    * (example @ https://video.etherpad.org theme)
    * not enough contrast on style buttons
    * default text is too bright, needs to be darker

¶* WEIRD CURSOR BEHAVIOURS on wrapped line:
    * from below going up - does not end up on end
    * gets stuck (but only on first character!)
    * [...]

¶* missing: stable link to latest version of documentation:
    I suggest it would be good to point https://etherpad.org/doc either to:
    a) an index of docs <--
    b) or the last version of them

¶* 100% cpu, then FAILURE (via timeouts) on moving bigger pads:
    My instance has ___ mb mysql DB.
    Pad in question has ___ edits (seen with most bigger pads).
    Command (via Curl) fails after ___ seconds.
    Log (on "DEBUG" says) ___
    ________________
    (more below!)

¶* you have space indented lines:
    (like this ... select whole line + copy + position to start of line + paste)
    it pastes, then gets merged on end of line
    ___________
    UPDATE: 20210119 cannot reproduce

¶* you're on a tabulator indented line & line above it is empty:
    press backspace.
    instead of deleting the indentation, it deletes the empty line above it.
    (this makes it difficult to clean random indents)
    ___________
    UPDATE: 20210119 can't repro

¶* loading delay in timeslider view, without any indication:
    on long documents, it takes maybe a minute (or more?) to load the revisions
    until then, the playback is rolling. but it's all foo.
    A) freeze playback until it's loaded
    B) indicate progress of loading
    C) ???

¶* connection failures (with lost content) happen all the time:
    * reconnect, i haven't seen working once ... it's a sign the thing will fail.
    * how to have an updated status if still connected?:
        A) maintain "lest ping" timer, if not reached, have a yellow light somewhere. make this optional.
    * how to not lose content?
    what could be done about this?:
        A) work with local storage:
            * "base" version updates ought to be applied to
            * when reconnected, did anything change? nope? apply!
        B) [...]

¶* try copying in something from gnome terminal, as "Copy as HTML" (with right click):
    at least for some inpiuts (while testing with E2H_PSTE) it breaks etherpad when pasting ... :
        1) view is fucked
        2) you can still type but it's all fucked
    it seems it's because it all wraps into a <pre> window! and this isn't stripped
    ---------------
    NOT EASY TO REPRODUCE ?

¶* copying in a tab-space, it default to 8 characters:
    i propose:
        * defaulting to 4
        * making it a setting

¶* ... also tabulation should be (optionally, at least) spaces based:
tab + shift-tab should work to a pre-defined spacing level

¶* listallpads API should include time (and maybe revision no):
    https://etherpad.org/doc/v1.7.5/#index_listallpads
    -----------
    A) all include this info (for backwards compatibility, add opt-in flag)
    B) listallpads changed after T
    [...]

¶* very big pads fail to export .etherpad
(give "proxy error")
----------
workaround:
    it works as a localhost export!
    this means it's a proxy timeout problem!
so:
    find if this is documented yet
    and where to change it (in a sensible way)

¶* fail to "import" very big .etherpad files (30MB)
maybe same (timeouts) problem?
... check debug log!

¶* timeslider date format is weird:
MM/DD/YYYY ......... REALLY?!
WTF

¶* (2020/11) On pad disconnect (sleep→restore, momentary connection loss, ...), this scary message awaits you in a red tab
"""
ErrorId: X2XN0VHQ9x31mrLceaRL
URL: https://pad.totalism.org/p/hahaha
UserAgent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0
Error: socket.io connection error: {"type":"TransportError","description":{"isTrusted":true}}
at https://pad.totalism.org/javascripts/lib/ep_etherpad-lite/static/js/pad.js?callback=require.define&v=c3a14a55 at line 225
"""
This did not happen in the 2020/05 versions.
As a general reccomendation, I would prefer if these "red screens of death" wouldn't be displayed to the user.
I'm not sure what they want to achieve ... to "nag the admin"? They're already in the logs, right?
Or at least - make it configurable?

¶* you have to first make an edit to be able to see the "import" button
that's just ... wierd. right?

¶* etherpad (and also html export) fucks up on specific characters, ends URL/link
known examples:
    *,– (yeah a middle sized one), [...]
reproduce:
    """https:// miro.medium.com/max/700/1*TT5CJCmZ_-Kj8QZ06y2WEw.png"""
    (drop space)
(at least) "*" seems valid per:
    https://stackoverflow.com/questions/1547899/which-characters-make-a-url-invalid

¶* ... does same for " ' ", see
https://en.wikipedia.org/wiki/Gentlemen's_agreement

¶* there is no way to paste long strings without having them broken up at ??? character
(this was an issue when doing 🔗test-datapad-css !)

¶* inconsistent url interpretation between editor and html export:
    in pads "www.x" is already a link, also "www.x.whatever" or "www.blabla.com:80"
    but not in export html, where you need to add the protocol.
    "whatever://haha.com" nowhere
    but http://x in both

¶* you copy-paste a line that has some empty lines after it:
you will get a bonus of 2 empty lines!

¶* somebody is writing at the very end of the document. no empty line after it. you're unable to be adding something later:
    A) enable possibility to MOVE DOWN a single additional line below the last person's caret:
        """
        if OTHER PERSON CARET at end of docu (moving end forward)
        then DOWN at that line makes a new line + puts MY CARET there
        """
    B) always CREATE EMPTY NEW LINE if text in last line:
        there cannot be anything in last line, it will create another one
    C) ###

¶* undercores in pad names, are dropped from tab title

¶* timeslider export/settings buttons don't pop up on top
instead you have to scroll up to get them.
either fix their position, or scroll up to them when pressed.

¶* URLs ending in ")" bracket are not put together correctly

https://en.wikipedia.org/wiki/De_ou_par_Marcel_Duchamp_ou_Rrose_S%C3%A9lavy_(La_Bo%C3%AEte-en-valise)
https://en.wikipedia.org/wiki/PARC_(company)

¶* padnames convert "_" in tab titles to spaces

this is weird, especially if it is trailing!
@khg is same as @khg_ !

¶* splitting on long lines (seems at 2000 characters)

this doesn't work for things like base64 encoded audio/svg etc
not sure if it's COPYING IN or perhaps also/even COPYING OUT

¶cursed copy sequence

https://stackoverflow.com/questions/4842424/list-of-ansi-color-escape-sequences
copy-pasting this part under
A Handy Script to Remind Yourself
really messes it all up
* cursor jumps way out, behaves all wrong
* can't even undo!
"""
print "\033[39m\\033[49m - Reset colour"print "\\033[2K - Clear Line"print "\\033[<L>;<C>H OR \\033[<L>;<C>f puts the cursor at line L and column C."print "\\033[<N>A Move the cursor up N lines"print "\\033[<N>B Move the cursor down N lines"print "\\033[<N>C Move the cursor forward N columns"print "\\033[<N>D Move the cursor backward N columns"print "\\033[2J Clear the screen, move to (0,0)"print "\\033[K Erase to end of line"print "\\033[s Save cursor position"print "\\033[u Restore cursor position"print " "print "\\033[4m Underline on"print "\\033[24m Underline off"print "\\033[1m Bold on"print "\\033[21m Bold off"
"""

¶icon seems missing?

2022/02 yeah; I've noticed this confusion for a while but not why!

¶no-message pad crash on big copy

20220314 case!
~~(with yuan-vodstvo)~~

it keeps restarting.
nothing in JS console, neither POD logs...n

¶[k?] "bad requst" 400 (in console)

https://github.com/ether/etherpad-lite/issues/2318

¶[!!*] bug: rename of bigger pads fails (timeouts)

(20190710, and oldie)
in a really weird way?
just crashes instance?

case "huje" 20190710:
    * first failed + crashed instance:
        nothing in WARN-level logs !
    * second time ...:
        """
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>502 Proxy Error</title>
</head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid
response from an upstream server.<br />
The proxy server could not handle the request <em><a href="/api/1.2.13/movePad">GET /api/1.2.13/movePad</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
<hr>
<address>Apache/2.4.29 (Ubuntu) Server at pad.totalism.org Port 443</address>
</body></html>
        """
    * ... but i went to source e2h and it was not there
    * ... and destination was and nothing broke !
    * so WTF

__________ 202111 ___________

Had weirder issue with moving pr-101→101:
    it made 101, but preserved pr-101
    also, pr-101 gave errors when EXPORTING or TRYING TO PLAY

RENAMING...
http://localhost:9001/api/1.2.13/movePad?apikey=#################&sourceID=pr-101&destinationID=101

EXPORTING...
[2021-11-06 23:35:20.837] [INFO] console - Exporting pad "101" in etherpad format
[2021-11-06 23:35:33.250] [ERROR] console - TypeError: Cannot read property 'pool' of null
    at Object.exports.getPadRaw (/home/etherpad_20210824/src/node/utils/ExportEtherpad.js:41:15)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Object.exports.doExport (/home/etherpad_20210824/src/node/handler/ExportHandler.js:59:17)
    at async /home/etherpad_20210824/src/node/hooks/express/importexport.js:63:9

REPLAYING...
"""
[2021-11-06 23:39:21.136] [INFO] access - [LEAVE] pad:pr-101 socket:utQ_QAsm-uCD3MTmAAAG IP:ANONYMOUS authorID:a.6AvMVpaya96qj2dc
[2021-11-06 23:39:21.818] [INFO] access - [ENTER] pad:101 socket:6q5PFc-Xgp7BtEkKAAAK IP:ANONYMOUS authorID:a.6AvMVpaya96qj2dc
[2021-11-06 23:39:26.592] [INFO] access - [LEAVE] pad:101 socket:6q5PFc-Xgp7BtEkKAAAK IP:ANONYMOUS authorID:a.6AvMVpaya96qj2dc
[2021-11-06 23:39:27.221] [INFO] access - [ENTER] pad:101 socket:2VXtqVqNkmopaNlqAAAL IP:ANONYMOUS authorID:a.6AvMVpaya96qj2dc
[2021-11-06 23:39:29.927] [WARN] console - failed to compose cs in pad: 101 startrev: 12000 current rev: 12019
[2021-11-06 23:39:29.928] [ERROR] console - Error while handling a changeset request for 101 Error: Not a exports: null { start: 10000, granularity: 100, requestID: 5505 }
[2021-11-06 23:39:29.990] [WARN] console - failed to compose cs in pad: 101 startrev: 12100 current rev: 12101
[2021-11-06 23:39:30.050] [WARN] console - failed to compose cs in pad: 101 startrev: 12200 current rev: 12201
[2021-11-06 23:39:30.097] [WARN] console - failed to compose cs in pad: 101 startrev: 12300 current rev: 12301
[...]
[2021-11-06 23:39:32.048] [WARN] console - failed to compose cs in pad: 101 startrev: 16800 current rev: 16801
[2021-11-06 23:39:32.095] [WARN] console - failed to compose cs in pad: 101 startrev: 16900 current rev: 16901
[2021-11-06 23:39:32.130] [WARN] console - failed to compose cs in pad: 101 startrev: 17000 current rev: 17001
[2021-11-06 23:39:32.171] [WARN] console - failed to compose cs in pad: 101 startrev: 17100 current rev: 17101
"""

Wanted to report but there seems to be some new pull requests, so will first try to update and see!

¶* pgup is super fast. pgdn is super slow.
~~<20230420~~
weird!

¶* sync error on trying to undo clear authorship
~~<20230705~~
reproduce:
    write something below another author's text
    clear authorship of both
    undo
    boom, "sync error"

(could also be more generic but won't try now)

¶[!] BUG/CUTS— (Artefact copy-pasting this in)

http://euro.dell.com/si/en/dhs/Monitors/monitor_e178wfp/pd.aspx?refid=monitor_e178wfp&cs=sidhs1&s=dhs
Like "Features & Benefits" bullets

¶[r!] MAINTAIN— (PNPM upgrade)

"""
pnpm setup
pnpm add -g pnpm@latest
pnpm --version
"""

<------------------------------------- (new)+ BUGS ↑

¶*** FEATURE REQUESTS

¶* merge two pads & preserve history!:
    1) on existing pod
    2) etherpad export from pod 1 → import to pod 2
    ______________
    how:
        v1) just append it at end (while preserving timestamps). then the people can manually merge.
        v2) do more fancy stuff

¶* timeslider BLAME:
    * be able to see when some edit hapened
    * jump to that edit easily (and see if something was fucked around it)
    -------------
    Problem:
        You have line 1-10
        You remove 4-8
        How will this show up in blame??

¶* timeslider SEARCH:
* [...]

¶* "suggestions mode":
like we know from google docs

¶* "limited edit mode":
    where you can only:
        1) delete your own characters
        2) others just for typos (= 10 chars)
        3) easy to ban people
    kind of like "suggestins" but better

<------------------------------------- (new) FEATURES ↑

¶*** EFFECTIVE COLLABORATION ON PADS

Seems missing!

For Totalism.org patterns, see 🔗ethering !

¶*** TIPS

¶Running localhost Etherpad offline? disable "installDeps.sh" !

just throw this out of run.sh:
"""
# Prepare the environment
###DAVID WTF ################################# DISABLE THIS FOR LESS PROBLEMS
#src/bin/installDeps.sh "$@" || exit 1
"""
and we good.

--------------------------------------------------------------
[!!!→] BUGS– Possibly, etherpad should consider just running with what it has for local installations

¶500 internal error for proxypass ... this did it!

"""
sudo a2enmod proxy_wstunnel
#besides the more obvious
sudo a2enmod proxy
sudo a2enmod proxy_http
"""

h/t https://stackoverflow.com/questions/23931987/apache-proxy-no-protocol-handler-was-valid
ties to https://github.com/ether/etherpad-lite/wiki/How-to-put-Etherpad-Lite-behind-a-reverse-Proxy

do guides say this?
* https://feeding.cloud.geek.nz/posts/installing-etherpad-on-debianubuntu/ :
    no
* https://github.com/ether/etherpad-lite/wiki/How-to-put-Etherpad-Lite-behind-a-reverse-Proxy :
    (official)
    YES
* [...]

¶Troubleshooting: Instance won't start after upgrade
(on example of TLM, 20220202)

1) check logs (→all looked okay)
2) check settings.json & apache conf (→ok)
3) try deleting npm ep_* plugins (→didn't work)
4) manually remove "node_packages" directory:
(→TOTALISM and PADLAND were empty, while this one had some 3 weird ones)
(→it worked)

¶Upgrade Etherpad version

* update etherpad for instance
    * make copy of etherpad dir
    * chown -R as etherpad
    * git pull

----------------------------------------
* check node version
    ↓
    ↓ need higher
    ↓
    * disable all 3 deploys in crontab
    * psagi node + stop them
    ----------
    * $ apt remove nodejs
    * wget new from website
    * follow default answer:
        @ https://stackoverflow.com/questions/63312642/how-to-install-node-tar-xz-file-in-linux?answertab=scoredesc#tab-top
        ↓
        basically
        sudo cp -r directory_name/{bin,include,lib,share} /usr/
    ----------
    * node -v
    ↓
    ↓
    ↓
    OK, includes new npm version as well

* try running directly
    as 'etherpad' user
    $ bin/run.sh
    ↓
    if ok, then undisable (in crontab)
    ↓
    should restart by itself!

* inform users it's been update, and that they should keep an eye.

¶*** PAD

¶Etherpad API / alternative clients / editors
(<TS)

https://github.com/ether/etherpad-lite/wiki/Alternative-clients
https://github.com/ether/etherpad-lite/wiki/HTTP-API-client-libraries

the ep api only allows to set the whole pad text, no operational transforms
the mobile version makes it hard to past (on android), also:
- "One thing I hear from folks when I ask what problems they have with Etherpad is mobile support" → https://github.com/ether/etherpad-lite/issues/3315
"disconnects happen often"
the getText() function in the API returns the pad contents independently of the public status of the pad, differently from the gui:
- https://github.com/ether/etherpad-lite/issues/3600
to embed ep requires iframe solutions like:
- https://www.findbestopensource.com/product/ether-etherpad-lite-jquery-plugin

¶Etherpad alternatives: other collab online editors / libraries
(<TS)

https://www.aarongreenlee.com/blog/notes-web-based-text-editor-rich-text-code-structured-text/
https://www.findbestopensource.com/product/vileda-githerpad
https://gist.github.com/abhisheksliam/e2f4afe3f2bdddb8fe6be0bff4fd656d
https://gist.github.com/manigandham/65543a0bc2bf7006a487
https://gist.github.com/muxator/b7913aa2957286ad2f8cf9a303dec6ef
https://stedolan.github.io/jq/ - cli json reader
https://www.aarongreenlee.com/blog/notes-web-based-text-editor-rich-text-code-structured-text/
minimal pad https://github.com/derveloper/githerpad

¶[!!**o] EXPLORE— CHECK THESE PLUGINS
20191203

def:
    * https://www.npmjs.com/package/ep_mypads :
        users and permissions
        what FRAMA uses !!!
    * https://www.npmjs.com/package/ep_htaccess_auth :
        maybe simple permissions ?
    * https://github.com/ether/ep_who_did_what :
        ###
    * https://github.com/ether/ep_what_have_i_missed :
        ###
    * https://github.com/ether/ep_comments_page :
        ###
    * https://github.com/ether/ep_offline_edit :
        ###
    * https://github.com/ether/etherpad-cli-client

cool:
    * https://github.com/ether/ep_mediawiki
    * https://github.com/ether/ep_table_of_contents
    * https://www.npmjs.com/package/ep_insert_text :
        WOW - autocomplete from ext source!
    * https://www.npmjs.com/package/ep_author_neat :
        colour author names ???:
        with CSS - usefull for Q&A, #templates, etc
    * https://www.npmjs.com/package/ep_export_authors :
        ... another same one !!!
            #e2h #feature !!!!!!

interesting:
    * https://www.npmjs.com/package/ep_sociallinks
    * https://www.npmjs.com/package/ep_text_statistics

huuuuuuh:
    * https://www.npmjs.com/package/ep_element
    * https://www.npmjs.com/package/ep_reference

¶[!!!**] TOMERGE— My resources from "###linux"
#tomerge and publish !!!!!!

¶[!!**] PUBLISH— Publish this site, by linking it to...
:
    * etherpad wiki
    * etherpad mailing list
    * stackoverflow
    * [...]

¶[!!] MISSING— Instance "sanity check" resource

* DB health
* checking logs
* regular maintenance
* ###

¶[!!!**] CONTRIBUTE/PUBLISH— Finish and report the bugs & feature requests

yea

¶[~] TRICK— Check out "/health" endpoint

"""
New /health endpoint for getting information about Etherpad's health (see
draft-inadarei-api-health-check-06).
"""

Very subtle result, only gives version number (?! thought this was hidden) and "okay".
But could replace @@sync ...

¶[r!!*] (etherpad maintenance)

$
cp -a etherpad_xxxxxxxxx #......................... make copy
chown -R etherpad: etherpad_xxxxxxx
npm install -g npm@latest #......................... query online if it's really latest
#———————————
"""
su etherpad
cd etherpad_xxxxxxxx
rm -rf node_packages
cd src
rm package-lock.json
rm -rf node_packages
npm install
npm audit fix # still not, but okay
"""

https://bobbyhadz.com/blog/npm-audit-fix-not-workingv
painful ...

¶[!!s] Refresh Etherpad: Just start again :)
as done 20231004

# fresh git clone
# copy over settings.json & APIKEY.txt files
# you done?!!?

¶[!!**tt] Etherpad seems full of vulnerabilities, is behind on updating packages, running old npm?
~2023/10

https://github.com/ether/etherpad-lite/issues/3492

¶[!!!**] BUG— instability ~2024

Failed to apply USER_CHANGES from author ### on pad (socket ###) on pad ###: AssertionError [ERR_ASSERTION]: Expected values to be strictly equal
1962 !== 1963
at handleUserChanges (/DEPLOYS/etherpad/src/node/handler/PadMessageHandler.js:656:12)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

¶————— 20240227 version refresh (with npm/node version bumps!)

¶[!!!**] NEW PROBLEMS ... CONSTANT DROPS AGAIN?!
"""
[2024-02-28T05:22:40.096] [ERROR] socket.io - Error handling pad message from bUq-J-csgO2W0-NaAABj: Error: pre-CLIENT_READY message from IP ANONYMOUS: {
"""
↓
"""
etherpad [ERROR] socket.io - Error handling pad message from Error: pre-CLIENT_READY message from IP ANONYMOUS:
"""

¶[!!**$] DOCS— (Generally, there should be a "what's new" or "upgrade guide")

* a collaboratively edited document, for all known versions of breaking changes
* each issue should have clear status, and push to AUTO of course:
    A) MANUAL: figure this out yourself
    B) DETECTED: at least you'll get an ERROR/clear warning
    C) AUTO: mitigate/propose automagically
* ###

Examples:
    * settings.json : file logging (~2024/01)
    * ... : breaking change of "socketTransportProtocols" ~2024/02
    * node/npm requirement bumps!
    * ###

¶[a!!$] INSTALL— Setup Etherpad logging to file

(now not in settings.json anymore!)

$ systemctl edit --full etherpad-lite.service
↓
Under "[Service]"
↓
"""
##### 20240228 , via https://unix.stackexchange.com/questions/408929/systemd-how-to-redirect-stdout-to-logfile
StandardOutput=file:/DEPLOYS/etherpad/etherpad.log
"""
↓
(do this for each instance!!!)

¶[a!!!$] BUG!!/TOEMAIL/TROUBLES— Upgrading ~2024/2, pad loading fails because websockets line in settings.json
20240228
(with Apache reverse proxy)

MUST BE
"socketTransportProtocols" : ["websocket", "polling"],

WAS
###

¶[a!] BUG?/TROUBLES— (UPDATE PROBLEMS 20240228@luna and 20240414@local)

WHAT IT WAS:
    must update requirements for npm and node

WHAT IT SAYS...:
↓
"""
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/pnpm'
npm ERR! [Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/pnpm'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'mkdir',
npm ERR!   path: '/usr/lib/node_modules/pnpm'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
"""

____ SOLUTION:

    (@luna):
        ...not documented...
        [!z] (I did actually make "etherpad" user own "/usr/lib/node_modules", but PROBABLY not needed!)

    (@luna, →22.04):
        just run etherpad as root once!
        $ "/bin/run.sh --root"
        ----
        (also did this before but prob not relation)
        https://stackoverflow.com/questions/49679808/error-eacces-permission-denied-mkdir-usr-local-lib-node-modules-node-sass-b

_________

¶[!*t] BUG?/TROUBLES?— (More instability, not sure if temporary, or will persist and pester?)
ties to "TRANSPORT" ?

POSSIBLY CONSULT
* https://github.com/ether/etherpad-lite/issues/5799
* ###

CASE
"""
[2024-02-28T05:22:40.096] [ERROR] socket.io - Error handling pad message from bUq-J-csgO2W0-NaAABj: Error: pre-CLIENT_READY message from IP ANONYMOUS: {
"type": "COLLABROOM",
"component": "pad",
"data": {
    "type": "USER_CHANGES",
    "baseRev": 12242,
    "changeset": "Z:1akg>1|1pb=1aie*0|1+1$\n",
    "apool": {
      "numToAttrib": {
        "0": [
          "author",
          "a.79rxh7BgG0WmjsT9"
        ]
      },
      "nextNum": 1
    }
}
}
    at Object.exports.handleMessage (/DEPLOYS/etherpad20240228/src/node/handler/PadMessageHandler.ts:270:11)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at <anonymous> (/DEPLOYS/etherpad20240228/src/node/handler/SocketIORouter.ts:84:14)
"""

¶[rr!!*efo] ADMIN/MAINTAIN— Review changelogs! (Etherpad 2.0+)
aka "what's new"

https://github.com/ether/etherpad-lite/blob/develop/CHANGELOG.md

20230425

Starting Etherpad: Etherpad can now be started with a single command: pnpm run prod in the root directory.

¶[!!*oo] PLUGIN— Audio input via ep_padgpt

https://gitlab.cemea.org/mallette/ep_padgpt
"""
This plugin allows you to add live speech recognition to etherpad using vosk-server : your microphone audio is converted to text and directly inserted in a pad !
[...]
Running the following commands (warning, it will use around 10GB of RAM because of vosk-server) will get you a running instance of etherpad with the plugin installed and everything needed at https://localhost :
"""

¶[!!*o] PADLAND— Fix the nicer padland links

"Add this to allow nice urls, such as "https://etherpad.example.org/padname""
https://github.com/ether/etherpad-lite/wiki/How-to-put-Etherpad-Lite-behind-a-reverse-Proxy

¶[!!*ot] ACL/— Have pass

https://github.com/ether/etherpad-lite/wiki/How-to-put-Etherpad-Lite-behind-a-reverse-Proxy
"""
        <Location />
            AuthType Basic
            AuthName "Welcome to the domain.org Etherpad"
            AuthUserFile /path/to/svn.passwd
            AuthGroupFile /path/to/svn.group
            Require group etherpad
            ProxyPass http://localhost:9001/ retry=0 timeout=30
            ProxyPassReverse http://localhost:9001/
        </Location>
"""

        <Location />
            AuthType Basic
            AuthName "Welcome to the domain.org Etherpad"
            AuthUserFile /path/to/svn.passwd
            AuthGroupFile /path/to/svn.group
            Require group etherpad
            ProxyPass http://localhost:9001/ retry=0 timeout=30
            ProxyPassReverse http://localhost:9001/
        </Location>

https://www.google.com/search?client=firefox-b-d&q=authtype+digest

¶[!!!**t] BUG/INSTALL/UNSTABLE— Frequent disconnect problems

20240403 : asked on Discord

https://github.com/ether/etherpad-lite/issues/2097
###

(via discord)
"""
Websocket is the best connection type as the connection is once established and will be kept alive as long as the user does not refresh the pad. Long polling means that there is every few seconds a request to the backend checking if the pad is up to date. This can have performance issues on larger instances.
"""

___________________________________________________________________

Closing @ 20240517 ... will try to wait/live with it for now

google: etherpad "xhr poll error"
https://stackoverflow.com/questions/45955032/constantly-reconnecting-to-websocket-session-id-unknown-etherpad
google: etherpad ws wss
https://github.com/ether/etherpad-lite/issues/2318
https://github.com/ether/etherpad-lite/issues/2318
https://gitlab.cemea.org/mallette/ep_padgpt/-/blob/main/static/js/client-hooks.js?ref_type=heads#L181-198
https://socket.io/docs/v4/client-options/#transports
https://stackoverflow.com/questions/41381444/websocket-connection-failed-error-during-websocket-handshake-unexpected-respon
https://firefox-source-docs.mozilla.org/devtools-user/network_monitor/inspecting_web_sockets/index.html
https://github.com/socketio/socket.io/issues/991

¶[!!**et] BUG— Scrub the site of "npm install"

https://github.com/ether/etherpad-lite/wiki/Available-Plugins
https://etherpad.org/doc/v2.0.3/#index_hooks

also
https://github.com/ether/etherpad-lite/wiki/Plugin-Wishlist ("no permissions") = dead link

¶[!!**eef] DEV/SETTINGS— Specify date format for "play" mode :)

Yup!

<---------------------------------- (new)++((new))

ETHERPAD

Edit Site

Edit CSS