pyspark.sql.streaming.DataStreamReader.option¶

DataStreamReader.option(key, value)[source]¶

Adds an input option for the underlying data source.

You can set the following option(s) for reading files:

timeZone: sets the string that indicates a time zone ID to be used to parse
timestamps in the JSON/CSV datasources or partition values. The following formats of timeZone are supported:
- Region-based zone ID: It should have the form ‘area/city’, such as ‘America/Los_Angeles’.
- Zone offset: It should be in the format ‘(+|-)HH:mm’, for example ‘-08:00’ or ‘+01:00’. Also ‘UTC’ and ‘Z’ are supported as aliases of ‘+00:00’.
Other short names like ‘CST’ are not recommended to use because they can be ambiguous. If it isn’t set, the current value of the SQL config spark.sql.session.timeZone is used by default.

New in version 2.0.0.

Notes

This API is evolving.

Examples

>>> s = spark.readStream.option("x", 1)