commit e09c96a75edf81b3d219223b6d3564ad7df10004
parent 3ef73a907e65cad2a250e2e9d345c58f19ae4ca0
Author: Hiltjo Posthuma <hiltjo@codemadness.org>
Date: Fri, 1 Jan 2021 22:36:18 +0100
README: add text about page redirects + tweak some words.
... move sections around in a more logical order and tweak some words.
Prompted by a question and feedback from Aleksei, thanks!
Diffstat:
| M | README | | | 42 | ++++++++++++++++++++++++++++-------------- |
1 file changed, 28 insertions(+), 14 deletions(-)
diff --git a/README b/README
@@ -316,18 +316,6 @@ advertisements, strip tracking parameters and more.
- - -
-The fetch function can be overridden in your sfeedrc file. This allows to
-replace the default curl(1) for sfeed_update with any other client to fetch the
-RSS/Atom data:
-
- # fetch a feed via HTTP/HTTPS etc.
- # fetch(name, url, feedfile)
- fetch() {
- hurl -m 1048576 -t 15 "$2" 2>/dev/null
- }
-
-- - -
-
Aggregate feeds. This filters new entries (maximum one day old) and sorts them
by newest first. Prefix the feed name in the title. Convert the TSV output data
to an Atom XML feed (again):
@@ -552,6 +540,18 @@ Now you can view feeds in mutt(1) for example.
- - -
+The fetch function can be overridden in your sfeedrc file. This allows to
+replace the default curl(1) for sfeed_update with any other client to fetch the
+RSS/Atom data or change the default curl options:
+
+ # fetch a feed via HTTP/HTTPS etc.
+ # fetch(name, url, feedfile)
+ fetch() {
+ hurl -m 1048576 -t 15 "$2" 2>/dev/null
+ }
+
+- - -
+
Incremental data updates using If-Modified-Since
For servers that support it some incremental updates and bandwidth-saving can
@@ -593,11 +593,11 @@ the --etag-save and --etag-compare options:
}
This comes at a cost of some privacy. For example there can be a unique
-generated ETag.
+generated ETag to pin and fingerprint a client.
- - -
-CDN's blocking requests due to missing User-Agent
+CDN's blocking requests due to a missing HTTP User-Agent request header
sfeed_update will not send the "User-Agent" header by default for privacy
reasons. Some CDNs like Cloudflare don't like this and will block such HTTP
@@ -612,6 +612,20 @@ Firefox 78.
- - -
+Page redirects
+
+For security and efficiency reasons by default redirects are not allowed and
+are treated as an error.
+
+For example to prevent hijacking an unencrypted http:// to https:// redirect or
+to not add time of an unnecessary page redirect each time. It is encouraged to
+use the final redirected url in the sfeedrc config file.
+
+If you want to ignore this advise you can override the fetch() function in the
+sfeedrc file and change the curl options "-L --max-redirs 0".
+
+- - -
+
Shellscript to export existing newsboat cached items from sqlite3 to the sfeed
TSV format.
