2026-09-27 12:37:16 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 502 Bad Gateway 2026-09-27 12:37:22 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): DNS lookup failed: no results for hostname lookup: yaldi.rideatom.com. 2026-09-27 12:37:22 [scrapy.core.scraper] ERROR: Error downloading Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 55, in wrap_twisted_exceptions yield File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 140, in download_request return await maybe_deferred_to_future(agent.download_request(request)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1092, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/endpoints.py", line 1091, in startConnectionAttempts raise error.DNSLookupError( twisted.internet.error.DNSLookupError: DNS lookup failed: no results for hostname lookup: yaldi.rideatom.com. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/engine.py", line 514, in _download result = yield self.downloader.fetch(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 226, in fetch result: Response | Request = yield ( ^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1257, in adapt extracted: _SelfResultT | Failure = result.result() ^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 87, in download_async result = await self._process_exception(ex, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 156, in _process_exception raise exception File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 80, in download_async result: Response | Request = await self._process_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 114, in _process_request return await download_func(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 281, in _enqueue_request return await maybe_deferred_to_future(d) # fired in _wait_for_download() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 348, in _wait_for_download response = await self._download(slot, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 318, in _download response: Response = await self.handlers.download_request_async(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/__init__.py", line 168, in download_request_async return await handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy_zyte_api/handler.py", line 251, in download_request return await fallback_handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 139, in download_request with wrap_twisted_exceptions(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 63, in wrap_twisted_exceptions raise CannotResolveHostError(str(e)) from e scrapy.exceptions.CannotResolveHostError: DNS lookup failed: no results for hostname lookup: yaldi.rideatom.com. 2026-09-27 12:37:22 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 502 Bad Gateway 2026-09-27 12:37:27 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): DNS lookup failed: no results for hostname lookup: api.knotcity.io. 2026-09-27 12:37:27 [scrapy.core.scraper] ERROR: Error downloading Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 55, in wrap_twisted_exceptions yield File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 140, in download_request return await maybe_deferred_to_future(agent.download_request(request)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1092, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/endpoints.py", line 1091, in startConnectionAttempts raise error.DNSLookupError( twisted.internet.error.DNSLookupError: DNS lookup failed: no results for hostname lookup: api.knotcity.io. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/engine.py", line 514, in _download result = yield self.downloader.fetch(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 226, in fetch result: Response | Request = yield ( ^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1257, in adapt extracted: _SelfResultT | Failure = result.result() ^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 87, in download_async result = await self._process_exception(ex, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 156, in _process_exception raise exception File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 80, in download_async result: Response | Request = await self._process_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 114, in _process_request return await download_func(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 281, in _enqueue_request return await maybe_deferred_to_future(d) # fired in _wait_for_download() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 348, in _wait_for_download response = await self._download(slot, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 318, in _download response: Response = await self.handlers.download_request_async(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/__init__.py", line 168, in download_request_async return await handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy_zyte_api/handler.py", line 251, in download_request return await fallback_handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 139, in download_request with wrap_twisted_exceptions(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 63, in wrap_twisted_exceptions raise CannotResolveHostError(str(e)) from e scrapy.exceptions.CannotResolveHostError: DNS lookup failed: no results for hostname lookup: api.knotcity.io. 2026-09-27 12:37:32 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): DNS lookup failed: no results for hostname lookup: app.kotobike.jp. 2026-09-27 12:37:32 [scrapy.core.scraper] ERROR: Error downloading Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 55, in wrap_twisted_exceptions yield File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 140, in download_request return await maybe_deferred_to_future(agent.download_request(request)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1092, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/endpoints.py", line 1091, in startConnectionAttempts raise error.DNSLookupError( twisted.internet.error.DNSLookupError: DNS lookup failed: no results for hostname lookup: app.kotobike.jp. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/engine.py", line 514, in _download result = yield self.downloader.fetch(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 226, in fetch result: Response | Request = yield ( ^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1257, in adapt extracted: _SelfResultT | Failure = result.result() ^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 87, in download_async result = await self._process_exception(ex, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 156, in _process_exception raise exception File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 80, in download_async result: Response | Request = await self._process_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 114, in _process_request return await download_func(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 281, in _enqueue_request return await maybe_deferred_to_future(d) # fired in _wait_for_download() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 348, in _wait_for_download response = await self._download(slot, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 318, in _download response: Response = await self.handlers.download_request_async(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/__init__.py", line 168, in download_request_async return await handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy_zyte_api/handler.py", line 251, in download_request return await fallback_handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 139, in download_request with wrap_twisted_exceptions(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 63, in wrap_twisted_exceptions raise CannotResolveHostError(str(e)) from e scrapy.exceptions.CannotResolveHostError: DNS lookup failed: no results for hostname lookup: app.kotobike.jp. 2026-09-27 12:37:38 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): DNS lookup failed: no results for hostname lookup: gbfs.cogobikeshare.com. 2026-09-27 12:37:38 [scrapy.core.scraper] ERROR: Error downloading Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 55, in wrap_twisted_exceptions yield File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 140, in download_request return await maybe_deferred_to_future(agent.download_request(request)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1092, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/endpoints.py", line 1091, in startConnectionAttempts raise error.DNSLookupError( twisted.internet.error.DNSLookupError: DNS lookup failed: no results for hostname lookup: gbfs.cogobikeshare.com. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/engine.py", line 514, in _download result = yield self.downloader.fetch(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1853, in _inlineCallbacks result = context.run( ^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/python/failure.py", line 467, in throwExceptionIntoGenerator return g.throw(self.value.with_traceback(self.tb)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 226, in fetch result: Response | Request = yield ( ^^^^^^^ File "/.venv/lib/python3.11/site-packages/twisted/internet/defer.py", line 1257, in adapt extracted: _SelfResultT | Failure = result.result() ^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 87, in download_async result = await self._process_exception(ex, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 156, in _process_exception raise exception File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 80, in download_async result: Response | Request = await self._process_request( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/middleware.py", line 114, in _process_request return await download_func(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 281, in _enqueue_request return await maybe_deferred_to_future(d) # fired in _wait_for_download() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 348, in _wait_for_download response = await self._download(slot, request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/__init__.py", line 318, in _download response: Response = await self.handlers.download_request_async(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/__init__.py", line 168, in download_request_async return await handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy_zyte_api/handler.py", line 251, in download_request return await fallback_handler.download_request(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/core/downloader/handlers/http11.py", line 139, in download_request with wrap_twisted_exceptions(): File "/usr/local/lib/python3.11/contextlib.py", line 158, in __exit__ self.gen.throw(typ, value, traceback) File "/.venv/lib/python3.11/site-packages/scrapy/utils/_download_handlers.py", line 63, in wrap_twisted_exceptions raise CannotResolveHostError(str(e)) from e scrapy.exceptions.CannotResolveHostError: DNS lookup failed: no results for hostname lookup: gbfs.cogobikeshare.com. 2026-09-27 12:37:48 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 502 Bad Gateway 2026-09-27 12:37:49 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 503 Service Unavailable 2026-09-27 12:37:56 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 502 Bad Gateway 2026-09-27 12:37:56 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 502 Bad Gateway 2026-09-27 12:41:56 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2026-09-27 12:41:56 [gbfs] ERROR: Expecting value: line 1 column 1 (char 0) Traceback (most recent call last): File "/locations/spiders/gbfs.py", line 371, in get_next_json return response.json() ^^^^^^^^^^^^^^^ File "/.venv/lib/python3.11/site-packages/scrapy/http/response/text.py", line 100, in json self._cached_decoded_json = json.loads(self.body) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads return _default_decoder.decode(s) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) 2026-09-27 12:43:40 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2026-09-27 12:43:52 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 500 Internal Server Error 2026-09-27 12:50:15 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): 502 Bad Gateway