Repository navigation
POST requests to wfs server failing #348
Description
Activity
This is a WFS issue: replicated using curl:
curl -v -d "SERVICE=WFS&VERSION=2.0.0&REQUEST=DescribeFeatureType&typeNames=WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SP" https://openmaps.gov.bc.ca/geo/pub/wfs * Host openmaps.gov.bc.ca:443 was resolved. * IPv6: (none) * IPv4: 142.34.229.4 * Trying 142.34.229.4:443... * Connected to openmaps.gov.bc.ca (142.34.229.4) port 443 * ALPN: curl offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF * ALPN: server did not agree on a protocol. Uses default. * Server certificate: * subject: C=CA; ST=British Columbia; L=Victoria; O=Government of the Province of British Columbia; CN=*.data.gov.bc.ca * start date: Oct 1 15:16:12 2024 GMT * expire date: Oct 16 15:16:11 2025 GMT * subjectAltName: host "openmaps.gov.bc.ca" matched cert's "openmaps.gov.bc.ca" * issuer: C=US; O=Entrust, Inc.; OU=See www.entrust.net/legal-terms; OU=(c) 2012 Entrust, Inc. - for authorized use only; CN=Entrust Certification Authority - L1K * SSL certificate verify ok. * using HTTP/1.x > POST /geo/pub/wfs HTTP/1.1 > Host: openmaps.gov.bc.ca > User-Agent: curl/8.7.1 > Accept: */* > Content-Length: 124 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 124 bytes < HTTP/1.1 500 Internal Server Error < content-type: text/plain; charset=utf-8 < content-length: 21 < vary: Origin < vary: Access-Control-Request-Method < vary: Access-Control-Request-Headers < date: Wed, 18 Dec 2024 20:12:01 GMT < server: uvicorn < x-kong-upstream-latency: 2 < x-kong-proxy-latency: 1 < * Connection #0 to host openmaps.gov.bc.ca left intact Internal Server Error%
Can confirm no issues with GET requests made by Python bcdata.
$ bcdata dump -v WHSE_TANTALIS.TA_SCHOOL_DISTRICTS_SVW > t.geojson 2024-12-18 12:09:30,389:INFO:bcdata.wfs: Total features requested: 59 2024-12-18 12:09:40,982:INFO:bcdata.wfs: https://openmaps.gov.bc.ca/geo/pub/wfs?service=WFS&version=2.0.0&request=GetFeature&typeName=WHSE_TANTALIS.TA_SCHOOL_DISTRICTS_SVW&outputFormat=json&SRSNAME=EPSG%3A4326&count=59Reacted by Andy TeucherConfirms working with
GETviacurl:curl -v "https://openmaps.gov.bc.ca/geo/pub/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=DescribeFeatureType&typeNames=WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SP" * Host openmaps.gov.bc.ca:443 was resolved. * IPv6: (none) * IPv4: 142.34.229.4 * Trying 142.34.229.4:443... * Connected to openmaps.gov.bc.ca (142.34.229.4) port 443 * ALPN: curl offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF * ALPN: server did not agree on a protocol. Uses default. * Server certificate: * subject: C=CA; ST=British Columbia; L=Victoria; O=Government of the Province of British Columbia; CN=*.data.gov.bc.ca * start date: Oct 1 15:16:12 2024 GMT * expire date: Oct 16 15:16:11 2025 GMT * subjectAltName: host "openmaps.gov.bc.ca" matched cert's "openmaps.gov.bc.ca" * issuer: C=US; O=Entrust, Inc.; OU=See www.entrust.net/legal-terms; OU=(c) 2012 Entrust, Inc. - for authorized use only; CN=Entrust Certification Authority - L1K * SSL certificate verify ok. * using HTTP/1.x > GET /geo/pub/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=DescribeFeatureType&typeNames=WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SP HTTP/1.1 > Host: openmaps.gov.bc.ca > User-Agent: curl/8.7.1 > Accept: */* > * Request completely sent off < HTTP/1.1 200 OK < content-type: application/gml+xml; version=3.2 < transfer-encoding: chunked < vary: Origin < vary: Access-Control-Request-Method < vary: Access-Control-Request-Headers < content-disposition: inline; filename=pub-WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SP.xsd < date: Wed, 18 Dec 2024 20:14:39 GMT < x-kong-upstream-latency: 11 < x-kong-proxy-latency: 0 < <?xml version="1.0" encoding="UTF-8"?><xsd:schema xmlns:gml="http://www.opengis.net/gml/3.2" xmlns:pub="http://delivery.openmaps.gov.bc.ca/geo/" xmlns:wfs="http://www.opengis.net/wfs/2.0" xmlns:xsd="http://www.w3.org/2001/XMLSchema" elementFormDefault="qualified" targetNamespace="http://delivery.openmaps.gov.bc.ca/geo/"> <xsd:import namespace="http://www.opengis.net/gml/3.2" schemaLocation="https://openmaps.gov.bc.ca/geo/schemas/gml/3.2.1/gml.xsd"/> <xsd:complexType name="WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SPType"> <xsd:complexContent> <xsd:extension base="gml:AbstractFeatureType"> <xsd:sequence> <xsd:element maxOccurs="1" minOccurs="1" name="FIRE_NUMBER" nillable="false" type="xsd:string"/> <xsd:element maxOccurs="1" minOccurs="0" name="VERSION_NUMBER" nillable="true" type="xsd:decimal"/> <xsd:element maxOccurs="1" minOccurs="1" name="FIRE_YEAR" nillable="false" type="xsd:decimal"/> <xsd:element maxOccurs="1" minOccurs="0" name="FIRE_CAUSE" nillable="true" type="xsd:string"/> <xsd:element maxOccurs="1" minOccurs="0" name="FIRE_LABEL" nillable="true" type="xsd:string"/> <xsd:element maxOccurs="1" minOccurs="0" name="FIRE_SIZE_HECTARES" nillable="true" type="xsd:decimal"/> <xsd:element maxOccurs="1" minOccurs="0" name="SOURCE" nillable="true" type="xsd:string"/> <xsd:element maxOccurs="1" minOccurs="0" name="GPS_TRACK_DATE" nillable="true" type="xsd:date"/> <xsd:element maxOccurs="1" minOccurs="0" name="LOAD_DATE" nillable="true" type="xsd:date"/> <xsd:element maxOccurs="1" minOccurs="0" name="FIRE_DATE" nillable="true" type="xsd:date"/> <xsd:element maxOccurs="1" minOccurs="0" name="CREATION_METHOD" nillable="true" type="xsd:string"/> <xsd:element maxOccurs="1" minOccurs="0" name="FEATURE_CODE" nillable="true" type="xsd:string"/> <xsd:element maxOccurs="1" minOccurs="0" name="SHAPE" nillable="true" type="gml:GeometryPropertyType"/> <xsd:element maxOccurs="1" minOccurs="1" name="OBJECTID" nillable="false" type="xsd:decimal"/> <xsd:element maxOccurs="1" minOccurs="0" name="SE_ANNO_CAD_DATA" nillable="true" type="xsd:hexBinary"/> <xsd:element maxOccurs="1" minOccurs="0" name="FEATURE_AREA_SQM" nillable="true" type="xsd:decimal"/> <xsd:element maxOccurs="1" minOccurs="0" name="FEATURE_LENGTH_M" nillable="true" type="xsd:decimal"/> </xsd:sequence> </xsd:extension> </xsd:complexContent> </xsd:complexType> <xsd:element name="WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SP" substitutionGroup="gml:AbstractFeature" type="pub:WHSE_LAND_AND_NATURAL_RESOURCE.PROT_HISTORICAL_FIRE_POLYS_SPType"/> </xsd:schema> * Connection #0 to host openmaps.gov.bc.ca left intact
I don't think so - all tests pass.
I've opened a ticket with DSS 🤞
Reacted by Stephanie Hazlitt and Sam AlbersDSS has fixed the service,
POSTrequests now working again, verified inbcdataand on the command lineReacted by Alexandre BevingtonReacted by Simon Norris and Alex Chubatythank you for the quick fix!
@ateucher the issue persists for some requests but not others:
> bcdata::bcdc_get_data("0bc73892-e41f-41d0-8d8e-828c16139337") ## works > bcdata::bcdc_get_data("61044e1a-cd80-4ed6-9f95-907262b9910f") ## fails Error: There was an issue sending this WFS request 2 paginated requests issued > bcdata::bcdc_get_data("11277e35-d8be-47e4-bb1f-c38e393179c6") ## fails with more info Error: There was an issue sending this WFS request ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ Request: URL: https://openmaps.gov.bc.ca/geo/pub/wfs POST fields: SERVICE=WFS&VERSION=2.0.0&REQUEST=DescribeFeatureType&typeNames=WHSE_LAND_USE_PLANNING.RMP_LANDSCAPE_UNIT_SVW Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate Accept: application/json, text/xml, application/xml, */* User-Agent: https://github.com/bcgov/bcdata Response: status: HTTP/1.1 500 Internal Server Error content-type: application/json transfer-encoding: chunked vary: Origin vary: Access-Control-Request-Method vary: Access-Control-Request-Headers x-kong-upstream-latency: 30002 x-kong-proxy-latency: 0
@achubaty I cannot reproduce the error
bcdata::bcdc_query_geodata("61044e1a-cd80-4ed6-9f95-907262b9910f") #> Querying 'natural-disturbance-type-map' record #> • Using collect() on this object will return 15666 features and 8 fields #> • Accessing this record requires pagination and will make 2 separate #> • requests to the WFS. See ?bcdc_options #> • At most six rows of the record are printed here #> ──────────────────────────────────────────────────────────────────────────────── #> Simple feature collection with 6 features and 8 fields #> Geometry type: POLYGON #> Dimension: XY #> Bounding box: xmin: 902336.9 ymin: 995587.5 xmax: 953437.5 ymax: 1255038 #> Projected CRS: NAD83 / BC Albers #> # A tibble: 6 × 9 #> id FEATURE_CLASS_SKEY NATURAL_DISTURBANCE_…¹ NATURAL_DISTURBANCE_…² #> <chr> <int> <chr> <chr> #> 1 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 2 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 3 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 4 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 5 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 6 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> # ℹ abbreviated names: ¹NATURAL_DISTURBANCE_TYPE_CODE, #> # ²NATURAL_DISTURBANCE_TYPE_DESC #> # ℹ 5 more variables: FEATURE_AREA_SQM <dbl>, FEATURE_LENGTH_M <dbl>, #> # OBJECTID <int>, SE_ANNO_CAD_DATA <chr>, geometry <POLYGON [m]>
> session_info() ─ Session info ───────────────────────────────────────── setting value version R version 4.4.2 (2024-10-31) os macOS Sonoma 14.4.1 system aarch64, darwin20 ui RStudio language (EN) collate en_US.UTF-8 ctype en_US.UTF-8 tz America/Vancouver date 2024-12-19 rstudio 2024.12.0+467 Kousa Dogwood (desktop) pandoc 3.2 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/aarch64/ (via rmarkdown) ─ Packages ───────────────────────────────────────────── package * version date (UTC) lib source bcdata * 0.5.0 2024-12-12 [1] CRAN (R 4.4.1)Sorry, the
bcdc_query_geodata()call works but not the corresponding call tobcdc_get_data(). Apologies for the copy-paste error in my post above.Sorry, second example also not working for me:
> bcdata::bcdc_get_data("11277e35-d8be-47e4-bb1f-c38e393179c6") Error: There was an issue sending this WFS request ▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄▄ Request: URL: https://openmaps.gov.bc.ca/geo/pub/wfs POST fields: SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&outputFormat=application%2Fjson&typeNames=WHSE_LAND_USE_PLANNING.RMP_LANDSCAPE_UNIT_SVW&SRSNAME=EPSG%3A3005 Content-Type: application/x-www-form-urlencoded Accept-Encoding: gzip, deflate Accept: application/json, text/xml, application/xml, */* User-Agent: https://github.com/bcgov/bcdata Response: status: HTTP/1.1 504 Gateway Time-out date: Thu, 19 Dec 2024 18:16:38 GMT content-type: application/json; charset=utf-8 transfer-encoding: chunked x-kong-upstream-latency: 60002 x-kong-proxy-latency: 1I have re-opened the ticket with the bcgov Data Systems & Services team as this issue is with the WFS.
Reacted by Alex ChubatyAll three examples work using
bcdata::bcdc_query_geodata()bcdata::bcdc_query_geodata("0bc73892-e41f-41d0-8d8e-828c16139337") #> You are accessing a simplified view of the data - see the catalogue record for details. #> Querying 'natural-resource-nr-district' record #> • Using collect() on this object will return 23 features and 12 fields #> • At most six rows of the record are printed here #> ──────────────────────────────────────────────────────────────────────────────── #> Simple feature collection with 6 features and 12 fields #> Geometry type: POLYGON #> Dimension: XY #> Bounding box: xmin: 831051 ymin: 455592.3 xmax: 1556586 ymax: 1522387 #> Projected CRS: NAD83 / BC Albers #> # A tibble: 6 × 13 #> id DISTRICT_NAME ORG_UNIT ORG_UNIT_NAME REGION_ORG_UNIT #> <chr> <chr> <chr> <chr> <chr> #> 1 WHSE_ADMIN_BOUNDARIES.AD… Peace Natura… DPC Peace Natura… RNO #> 2 WHSE_ADMIN_BOUNDARIES.AD… Cascades Nat… DCS Cascades Nat… RTO #> 3 WHSE_ADMIN_BOUNDARIES.AD… Cariboo-Chil… DCC Cariboo-Chil… RCB #> 4 WHSE_ADMIN_BOUNDARIES.AD… Sunshine Coa… DSC Sunshine Coa… RSC #> 5 WHSE_ADMIN_BOUNDARIES.AD… Okanagan Shu… DOS Okanagan Shu… RTO #> 6 WHSE_ADMIN_BOUNDARIES.AD… Stuart Necha… DVA Stuart Necha… ROM #> # ℹ 8 more variables: REGION_ORG_UNIT_NAME <chr>, FEATURE_CODE <chr>, #> # FEATURE_NAME <chr>, OBJECTID <int>, SE_ANNO_CAD_DATA <chr>, #> # FEATURE_AREA_SQM <dbl>, FEATURE_LENGTH_M <dbl>, geometry <POLYGON [m]> bcdata::bcdc_query_geodata("61044e1a-cd80-4ed6-9f95-907262b9910f") #> Querying 'natural-disturbance-type-map' record #> • Using collect() on this object will return 15666 features and 8 fields #> • Accessing this record requires pagination and will make 2 separate #> • requests to the WFS. See ?bcdc_options #> • At most six rows of the record are printed here #> ──────────────────────────────────────────────────────────────────────────────── #> Simple feature collection with 6 features and 8 fields #> Geometry type: POLYGON #> Dimension: XY #> Bounding box: xmin: 902336.9 ymin: 995587.5 xmax: 953437.5 ymax: 1255038 #> Projected CRS: NAD83 / BC Albers #> # A tibble: 6 × 9 #> id FEATURE_CLASS_SKEY NATURAL_DISTURBANCE_…¹ NATURAL_DISTURBANCE_…² #> <chr> <int> <chr> <chr> #> 1 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 2 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 3 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 4 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 5 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> 6 WHSE_FOREST_… 435 NDT2 Ecosystems with infre… #> # ℹ abbreviated names: ¹NATURAL_DISTURBANCE_TYPE_CODE, #> # ²NATURAL_DISTURBANCE_TYPE_DESC #> # ℹ 5 more variables: FEATURE_AREA_SQM <dbl>, FEATURE_LENGTH_M <dbl>, #> # OBJECTID <int>, SE_ANNO_CAD_DATA <chr>, geometry <POLYGON [m]> bcdata::bcdc_query_geodata("11277e35-d8be-47e4-bb1f-c38e393179c6") #> Querying 'landscape-units-of-british-columbia-current' record #> • Using collect() on this object will return 1922 features and 20 fields #> • At most six rows of the record are printed here #> ──────────────────────────────────────────────────────────────────────────────── #> Simple feature collection with 6 features and 20 fields #> Geometry type: POLYGON #> Dimension: XY #> Bounding box: xmin: 273374.5 ymin: 1562600 xmax: 715407.1 ymax: 1735705 #> Projected CRS: NAD83 / BC Albers #> # A tibble: 6 × 21 #> id LANDSCAPE_UNIT_ID LANDSCAPE_UNIT_PROVID LANDSCAPE_UNIT_NUMBER #> <chr> <int> <int> <chr> #> 1 WHSE_LAND_USE_P… 1 1256 <NA> #> 2 WHSE_LAND_USE_P… 2 712 <NA> #> 3 WHSE_LAND_USE_P… 3 1324 <NA> #> 4 WHSE_LAND_USE_P… 4 45 <NA> #> 5 WHSE_LAND_USE_P… 5 468 <NA> #> 6 WHSE_LAND_USE_P… 6 1268 <NA> #> # ℹ 17 more variables: LANDSCAPE_UNIT_NAME <chr>, #> # BIODIVERSITY_EMPHASIS_OPTION <chr>, BEO_SUB_TYPE_APPLICABLE <chr>, #> # ORIGINAL_DECISION_DATE <chr>, LAST_AMENDMENT_DATE <chr>, #> # GIS_CHANGE_DATE <date>, GIS_CHANGE_PERSON <chr>, CHANGE_REASON <chr>, #> # INITIATOR_OF_CHANGE <chr>, ENABLING_DOCUMENT_TITLE <chr>, #> # ENABLING_DOCUMENT_URL <chr>, FEATURE_CODE <chr>, FEATURE_AREA_SQM <dbl>, #> # FEATURE_LENGTH_M <dbl>, OBJECTID <int>, SE_ANNO_CAD_DATA <chr>, …
Created on 2024-12-19 with reprex v2.0.2
Sorry, the
bcdc_query_geodata()call works but not the corresponding call tobcdc_get_data(). Apologies for the copy-paste error in my post above.The
bcdc_get_data()example is working for me, have you updated yourbcdatapackage to the new release @achubaty ?bcdata::bcdc_get_data("0bc73892-e41f-41d0-8d8e-828c16139337") #> You are accessing a simplified view of the data - see the catalogue record for details. #> Simple feature collection with 23 features and 12 fields #> Geometry type: MULTIPOLYGON #> Dimension: XY #> Bounding box: xmin: 273366.8 ymin: 359771.8 xmax: 1870587 ymax: 1735721 #> Projected CRS: NAD83 / BC Albers #> # A tibble: 23 × 13 #> id DISTRICT_NAME ORG_UNIT ORG_UNIT_NAME REGION_ORG_UNIT #> * <chr> <chr> <chr> <chr> <chr> #> 1 WHSE_ADMIN_BOUNDARIES.A… Peace Natura… DPC Peace Natura… RNO #> 2 WHSE_ADMIN_BOUNDARIES.A… Cascades Nat… DCS Cascades Nat… RTO #> 3 WHSE_ADMIN_BOUNDARIES.A… Cariboo-Chil… DCC Cariboo-Chil… RCB #> 4 WHSE_ADMIN_BOUNDARIES.A… Sunshine Coa… DSC Sunshine Coa… RSC #> 5 WHSE_ADMIN_BOUNDARIES.A… Okanagan Shu… DOS Okanagan Shu… RTO #> 6 WHSE_ADMIN_BOUNDARIES.A… Stuart Necha… DVA Stuart Necha… ROM #> 7 WHSE_ADMIN_BOUNDARIES.A… Quesnel Natu… DQU Quesnel Natu… RCB #> 8 WHSE_ADMIN_BOUNDARIES.A… Chilliwack N… DCK Chilliwack N… RSC #> 9 WHSE_ADMIN_BOUNDARIES.A… Fort Nelson … DFN Fort Nelson … RNO #> 10 WHSE_ADMIN_BOUNDARIES.A… Prince Georg… DPG Prince Georg… ROM #> # ℹ 13 more rows #> # ℹ 8 more variables: REGION_ORG_UNIT_NAME <chr>, FEATURE_CODE <chr>, #> # FEATURE_NAME <chr>, OBJECTID <int>, SE_ANNO_CAD_DATA <chr>, #> # FEATURE_AREA_SQM <dbl>, FEATURE_LENGTH_M <dbl>, geometry <MULTIPOLYGON [m]>
Yes, I am using v0.5.0.
All three examples work with
bcdc_query_geodata()alone, but the latter 2 fail with additionalcollect()call. All three examples fail usingbcdc_get_data()calls.> bcdata::bcdc_get_data("0bc73892-e41f-41d0-8d8e-828c16139337") ## works > bcdata::bcdc_get_data("61044e1a-cd80-4ed6-9f95-907262b9910f") ## fails > bcdata::bcdc_get_data("11277e35-d8be-47e4-bb1f-c38e393179c6") ## fails
Reacted by Stephanie HazlittIt looks like it is still an issue with POST requests, but only those requiring pagination using
sortby,countandstartIndexparameters:Works with GET:
$ curl https://openmaps.gov.bc.ca/geo/pub/wfs?SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&outputFormat=application%2Fjson&typeNames=WHSE_FOREST_VEGETATION.BEC_NATURAL_DISTURBANCE_SV&SRSNAME=EPSG%3A3005&sortby=OBJECTID&startIndex=0&count=10000 -o bec1.jsonNot with POST:
$ curl -v -d "SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&outputFormat=application%2Fjson&typeNames=WHSE_FOREST_VEGETATION.BEC_NATURAL_DISTURBANCE_SV&SRSNAME=EPSG%3A3005&sortby=OBJECTID&startIndex=0&count=10000" https://openmaps.gov.bc.ca/geo/pub/wfs * Host openmaps.gov.bc.ca:443 was resolved. * IPv6: (none) * IPv4: 142.34.229.4 * Trying 142.34.229.4:443... * Connected to openmaps.gov.bc.ca (142.34.229.4) port 443 * ALPN: curl offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF * ALPN: server did not agree on a protocol. Uses default. * Server certificate: * subject: C=CA; ST=British Columbia; L=Victoria; O=Government of the Province of British Columbia; CN=*.data.gov.bc.ca * start date: Oct 1 15:16:12 2024 GMT * expire date: Oct 16 15:16:11 2025 GMT * subjectAltName: host "openmaps.gov.bc.ca" matched cert's "openmaps.gov.bc.ca" * issuer: C=US; O=Entrust, Inc.; OU=See www.entrust.net/legal-terms; OU=(c) 2012 Entrust, Inc. - for authorized use only; CN=Entrust Certification Authority - L1K * SSL certificate verify ok. * using HTTP/1.x > POST /geo/pub/wfs HTTP/1.1 > Host: openmaps.gov.bc.ca > User-Agent: curl/8.7.1 > Accept: */* > Content-Length: 197 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 197 bytes < HTTP/1.1 500 Internal Server Error < content-type: application/json < transfer-encoding: chunked < vary: Origin < vary: Access-Control-Request-Method < vary: Access-Control-Request-Headers < x-kong-upstream-latency: 24408 < x-kong-proxy-latency: 0 < * Connection #0 to host openmaps.gov.bc.ca left intact {"message":"Upstream service error - An unexpected error occurred","upstream":{"requestId":"6043c512-1010262","timestamp":"2024-12-19T20:00:12.800+00:00","path":"/geo/pub/wfs","status":500,"error":"Internal Server Error"}}%It looks like it is timing out with bigger requests.
This worked: count=100, startIndex=20
curl -d "SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&typeNames=WHSE_FOREST_VEGETATION.BEC_NATURAL_DISTURBANCE_SV&OutputFormat=application/json&sortby=OBJECTID&StartIndex=20&count=100" https://openmaps.gov.bc.ca/geo/pub/wfs > dev/nullThis timed out: count=5666, startIndex=10000
$ curl -v -d "SERVICE=WFS&VERSION=2.0.0&REQUEST=GetFeature&typeNames=WHSE_FOREST_VEGETATION.BEC_NATURAL_DISTURBANCE_SV&OutputFormat=application/json&sortby=OBJECTID&StartIndex=10000&count=5666" https://openmaps.gov.bc.ca/geo/pub/wfs > dev/null * Host openmaps.gov.bc.ca:443 was resolved. * IPv6: (none) * IPv4: 142.34.229.4 * Trying 142.34.229.4:443... * Connected to openmaps.gov.bc.ca (142.34.229.4) port 443 * ALPN: curl offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256 / [blank] / UNDEF * ALPN: server did not agree on a protocol. Uses default. * Server certificate: * subject: C=CA; ST=British Columbia; L=Victoria; O=Government of the Province of British Columbia; CN=*.data.gov.bc.ca * start date: Oct 1 15:16:12 2024 GMT * expire date: Oct 16 15:16:11 2025 GMT * subjectAltName: host "openmaps.gov.bc.ca" matched cert's "openmaps.gov.bc.ca" * issuer: C=US; O=Entrust, Inc.; OU=See www.entrust.net/legal-terms; OU=(c) 2012 Entrust, Inc. - for authorized use only; CN=Entrust Certification Authority - L1K * SSL certificate verify ok. * using HTTP/1.x > POST /geo/pub/wfs HTTP/1.1 > Host: openmaps.gov.bc.ca > User-Agent: curl/8.7.1 > Accept: */* > Content-Length: 178 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 178 bytes < HTTP/1.1 504 Gateway Time-out < date: Thu, 19 Dec 2024 20:18:44 GMT < content-type: application/json; charset=utf-8 < transfer-encoding: chunked < x-kong-upstream-latency: 60002 < x-kong-proxy-latency: 1 < * Connection #0 to host openmaps.gov.bc.ca left intact {"message":"Upstream service error - The upstream server is timing out","upstream":{"message":"The upstream server is timing out"}}%This appears to be fixed on the server side now, @achubaty, @bevingtona if you don't mind doing a quick test that would be great!
Looks like it's working again (albeit slowly). Thank you!
Yeah, I found it a bit slow as well... thanks for testing!
Originally posted by @achubaty in #347