Dev apicalls

.Rbuildignore

@@ -48,3 +48,4 @@ paper.*
 ^vignettes/images/
 ^vignettes/data/
 ^data-raw/
+^\.lintr$

.lintr

@@ -0,0 +1,17 @@
+exclusions: list(
+    "tests/testthat/fixtures",
+    "R/get_sites.R" = list(commented_code_linter = 80),
+    "R/get_datasets.R" = list(commented_code_linter = 94),
+    "R/get_downloads.R" = list(commented_code_linter = 76),
+    "R/cleanNULL.R" = list(object_name_linter = Inf),
+    "R/parseLocation.R" = list(object_name_linter = Inf),
+    "R/parseURL.R" = list(object_name_linter = Inf),
+    "R/pingNeotoma.r" = list(object_name_linter = Inf),
+    "R/pubAuthors.R" = list(object_name_linter = Inf),
+    "R/testNull.R" = list(object_name_linter = Inf),
+    "R/toWide.R" = list(object_name_linter = Inf),
+    "R/set_contact.R" = list(object_name_linter = Inf),
+    "R/site-methods.R" = list(object_usage_linter = Inf),
+    "R/chronology-methods.R" = list(object_usage_linter = Inf),
+    "data-raw/prepare_vignette_data.R" = list(object_usage_linter = Inf)
+  )

NAMESPACE

@@ -159,6 +159,7 @@ importFrom(httr,GET)
 importFrom(httr,HEAD)
 importFrom(httr,VERB)
 importFrom(httr,add_headers)
+importFrom(httr,config)
 importFrom(httr,content)
 importFrom(httr,http_error)
 importFrom(httr,modify_url)
@@ -206,7 +207,6 @@ importFrom(sf,st_transform)
 importFrom(sf,st_union)
 importFrom(stats,na.omit)
 importFrom(stringr,str_detect)
-importFrom(stringr,str_extract_all)
 importFrom(stringr,str_match)
 importFrom(stringr,str_match_all)
 importFrom(tidyr,pivot_wider)

NEWS.md

@@ -1,5 +1,50 @@
 # News neotoma2 R package
 
+## neotoma2 1.1.1
+
+* The default request timeout is 125 seconds, just above the 120 seconds at which the API gateway gives up, so the client and the gateway agree and a slow query  is reported once instead of being abandoned and repeated. `NEOTOMA_TIMEOUT` and `NEOTOMA_RETRIES` still override both.
+* Gentler on the Neotoma API:
+  * An HTTP 500 is no longer retried (it cannot succeed on a repeat), and a busy
+    gateway (502, 503, 504) is retried once, after a jittered wait of 30 to 60
+    seconds. A `Retry-After` header on a 429 or 503 is honoured.
+  * Requests identify the package: the User-Agent is now
+    `neotoma2/<version> (R <version>)`. Set `options(neotoma2.contact = "you@example.org")`
+    or the `NEOTOMA_CONTACT` environment variable to add a contact address.
+  * `get_table()` keeps its answer for the rest of the session (use `refresh = TRUE`
+    to ask again), and `get_downloads()` requests long vectors of dataset IDs in
+    batches of 25 and keeps the answers for the session. `options(neotoma2.cache = FALSE)`
+    turns the cache off.
+  * `get_downloads()` no longer repeats a failed request with its arguments removed.
+  * A long list of IDs for `data/taxa`, `data/publications` or other endpoints
+    without a POST route is now sent as several short `GET` requests, instead of a
+    `POST` the API answered with a redirect to its documentation.
+  * A long `get_downloads()` request that used `all_data = TRUE` sent a malformed
+    list of dataset IDs, which the API rejected with an HTTP 500.
+  * An answer that is a web page rather than data now raises a clear error.
+  * A failed request reports the reason the API gives (`The API said: ...`), in
+    both the current response shape (`data: null`, `message`) and the earlier one
+    (the error in `data`).
+* The development server is now reached over `https`. `http://api-dev...` redirects
+  to `https`, and the redirect turned a `POST` into a `GET` without its body, so a
+  spatial (`loc`) query sent with `set_server("dev")` came back as the unfiltered
+  first page of records. A `POST` now also keeps its method and body through any
+  redirect.
+* Long lists of IDs for `get_sites()`, `get_datasets()` and `get_downloads()` are
+  sent as `POST` requests in chunks of 100, as the API accepts them there, and a
+  long list of filter values (for example many taxon names) is sent as a `POST`
+  body instead of a URL too long to send. An answer that does not match the IDs
+  asked for raises an error instead of being returned.
+* `get_taxa()` was rewritten. `get_taxa(<id>)` used to ask for datasets with no
+  taxon filter at all; `get_taxa(taxa = )` passed a function in place of the names;
+  `all_data`, `datasettype`, `loc`, `limit` and other arguments were dropped or
+  ignored. It now forwards them, never requests datasets without a taxon filter,
+  and warns before sending more than 25 taxon names in one request.
+* `get_taxon(taxonname = "Pinus%")` now returns every page of matches (`all_data`
+  defaults to `TRUE`, as documented) instead of only the first, and a taxon record
+  without a `taxagroupid` no longer makes it fail. `get_taxon(numeric(0))` returns `NULL`.
+* `all_data` is now honoured in `get_datasets()` and `get_downloads()` when called on a
+  `sites` object; the argument was previously ignored.
+
 ## neotoma2 1.1.0
 
 * Added `count()` to tally sites, collection units, or datasets in a record object.
@@ -13,9 +58,6 @@
 * A request that times out is retried at most once. Retrying a slow query stacks
   work onto an API that is already struggling, which made spatial (`loc`) queries
   worse rather than better; other transient failures still retry as before.
-* Raised the default request timeout to 180 seconds, above the observed latency of
-  the spatial endpoints, so healthy-but-slow queries are no longer aborted
-  mid-flight. `NEOTOMA_TIMEOUT` and `NEOTOMA_RETRIES` still override both.
 * A spatial geometry is now converted once per query rather than once per page,
   and the API's parameter list is fetched once per session rather than on every
   `get_*()` call.