[Haskell-cafe] Parsing LocalTime from Unix seconds

[Marc Busqué]

In GHCi

```
:m +Data.Time
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe UTCTime
-- => Just 2018-08-31 03:00:06 UTC
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe LocalTime
-- => Just 1970-01-01 00:00:00
```

Why? ¯\(°_o)/¯

Marc Busqué
http://waiting-for-dev.github.io/about/

[Alexander V Vershilov]

Hi Marc,

The best way of answering such questions is to check the source code.
Hackage provides
a nice way of doing that - click on 'Source' near the instance that
you are interested in:

https://hackage.haskell.org/package/time-1.6.0.1/docs/Data-Time-Format.html#t:ParseTime

And you'll see the implementation

```

instance ParseTime LocalTime where
buildTime l xs = LocalTime <$> (buildTime l xs) <*> (buildTime l xs)
```

That builds time from `Day` and `TimeOfDay` passing your parse string
to each of those.
Then you can check ParseTime instance of Day:

https://hackage.haskell.org/package/time-1.6.0.1/docs/src/Data.Time.Format.Parse.html#line-331

I'm not providing it here, as it's quite big, but the main point is
that `s` is ignored so in that case
Day appear to be:

```
rest (YearMonth m:_) = let
d = safeLast 1 [x | MonthDay x <- cs]
in fromGregorianValid y m d
```
with y=m=d=1

if you continue the process for TimeOfDay you'll find that `s` is
ignored there as well, and
`midnight = TimeOfDay 0 0 0` is returned in that case.

So it appeared that LocalTime consists of the components that ignore
your parse string and return
default value instead.

I don't know if that is intended behaviour or not, but for me it makes
more sense to parse to UTCTime/POSIXTime
and then convert into LocalTime, in case if you get seconds as input.

Hope that helps.

Post by Marc Busqué
In GHCi
```
:m +Data.Time
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe UTCTime
-- => Just 2018-08-31 03:00:06 UTC
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe LocalTime
-- => Just 1970-01-01 00:00:00
```
Why? ¯\(°_o)/¯
Marc Busqué
http://waiting-for-dev.github.io/about/_______________________________________________
Haskell-Cafe mailing list
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

[Marc Busqué]

Thanks Alexander for your answer.

I opened an issue in `time` repository:

https://github.com/haskell/time/issues/104

It seems it is intended behaviour, but I think it is inconsistent and it
makes difficult to parse from a string when you don't know beforehand
the format it has.

In the issue link there is the developed version of my answer... :)

Marc Busqué
http://waiting-for-dev.github.io/about/

Post by Alexander V Vershilov
Hi Marc,
The best way of answering such questions is to check the source code.
Hackage provides
a nice way of doing that - click on 'Source' near the instance that
https://hackage.haskell.org/package/time-1.6.0.1/docs/Data-Time-Format.html#t:ParseTime
And you'll see the implementation
```
instance ParseTime LocalTime where
buildTime l xs = LocalTime <$> (buildTime l xs) <*> (buildTime l xs)
```
That builds time from `Day` and `TimeOfDay` passing your parse string
to each of those.
https://hackage.haskell.org/package/time-1.6.0.1/docs/src/Data.Time.Format.Parse.html#line-331
I'm not providing it here, as it's quite big, but the main point is
that `s` is ignored so in that case
```
rest (YearMonth m:_) = let
d = safeLast 1 [x | MonthDay x <- cs]
in fromGregorianValid y m d
```
with y=m=d=1
if you continue the process for TimeOfDay you'll find that `s` is
ignored there as well, and
`midnight = TimeOfDay 0 0 0` is returned in that case.
So it appeared that LocalTime consists of the components that ignore
your parse string and return
default value instead.
I don't know if that is intended behaviour or not, but for me it makes
more sense to parse to UTCTime/POSIXTime
and then convert into LocalTime, in case if you get seconds as input.
Hope that helps.

Post by Marc Busqué
In GHCi
```
:m +Data.Time
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe UTCTime
-- => Just 2018-08-31 03:00:06 UTC
parseTimeM True defaultTimeLocale "%s" "1535684406" :: Maybe LocalTime
-- => Just 1970-01-01 00:00:00
```
Why? ¯\(°_o)/¯
Marc Busqué
http://waiting-for-dev.github.io/about/_______________________________________________
Haskell-Cafe mailing list
http://mail.haskell.org/cgi-bin/mailman/listinfo/haskell-cafe
Only members subscribed via the mailman list are allowed to post.

--
Alexander

[Olaf Klinke]

Is the result of parsing Unix seconds to LocalTime even well-defined? A LocalTime could be in any time zone, while Unix epoch is relative to a certain time-point in UTC. Hence the parse result must depend on what time zone the LocalTime is referring to.

Olaf

[Tobias Dammers]

Post by Olaf Klinke
Is the result of parsing Unix seconds to LocalTime even well-defined?
A LocalTime could be in any time zone, while Unix epoch is relative to
a certain time-point in UTC. Hence the parse result must depend on
what time zone the LocalTime is referring to.

It's even worse - unix time may also refer to the host's local timezone,
although these days almost everything sets the RTC to UTC, and corrects
for local timezone on the fly (simply because this allows for somewhat
sane handling of DST and such).

Also, unix time may represent either actual seconds elapsed since epoch, or
"logical" seconds since epoch (ignoring leap seconds, such that midnight
is always a multiple of 86400 seconds).


However, once you pick a convention for the second part, you can convert
unix timestamps to some "local time without timezone info" data
structure (e.g. LocalTime); whether you assume UTC or any timezone
becomes relevant when you want to convert that data structure into a
data structure that does contain timezone info explicitly (like
ZonedTime) or implicitly (like UTCTime).

https://hackage.haskell.org/package/time-1.9.2/docs/Data-Time.html
provides a nice overview of the various date/time types defined in the
`time` package, and what they mean. The relevant ones here are:

- UTCTime: idealized (ignoring leap seconds) absolute moment in time
- LocalTime: moment in time, in some unspecified timezone
- ZonedTime: moment in time, in a specific timezone

And the "morally correct" way of parsing unix time into anything would
be to go through LocalTime first, then either convert to UTCTime if the
unix timestamp may be interpreted as being in UTC, or attaching the
correct timezone, yielding a ZonedTime. All this assuming that you can
afford to ignore leap seconds one way or another.

[Viktor Dukhovni]

Post by Tobias Dammers
Also, unix time may represent either actual seconds elapsed since epoch, or
"logical" seconds since epoch (ignoring leap seconds, such that midnight
is always a multiple of 86400 seconds).

That would be a violation of the specification:

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

On all extant systems Unix Time is based on an 86400-second day, regardless
of any leap seconds. The RTC clock has nothing to do with this, the epoch
time is defined as an interval. It only becomes fuzzy when leap seconds
are being handled, as different systems may handle the leap second in somewhat
different ways. Since the epoch time is quantized anyhow to a 1s granularity,
you can expect the epoch time reported by different systems to differ by +/-1s
normally, even with clocks reasonably well synchronized, and +/-2s when leap
seconds are being added if they're not both using the same adjustment algorithm
(say NTP leap second smearing).

[Tobias Dammers]

Post by Viktor Dukhovni

Post by Tobias Dammers
Also, unix time may represent either actual seconds elapsed since epoch, or
"logical" seconds since epoch (ignoring leap seconds, such that midnight
is always a multiple of 86400 seconds).

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

Ah yes, you are right. Problem is, while Unix systems to day use Unix
time this way, not all other software systems do, or they may assume
that "current unix timestamp - some unix timestamp in the past =
exact number of seconds elapsed between timestamps". Which doesn't hold
in the presence of leap seconds, no matter how you implement them. In
other words, people use Unix timestamps (both the word and actual
implementations) sloppily.

[Viktor Dukhovni]

Post by Tobias Dammers

Post by Viktor Dukhovni

Post by Tobias Dammers
Also, unix time may represent either actual seconds elapsed since epoch, or
"logical" seconds since epoch (ignoring leap seconds, such that midnight
is always a multiple of 86400 seconds).

http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap04.html#tag_04_16

Ah yes, you are right. Problem is, while Unix systems to day use Unix
time this way, not all other software systems do, or they may assume
that "current unix timestamp - some unix timestamp in the past =
exact number of seconds elapsed between timestamps". Which doesn't hold
in the presence of leap seconds, no matter how you implement them. In
other words, people use Unix timestamps (both the word and actual
implementations) sloppily.

That may be so, but Haskell's libraries should not cater to broken implementations.

UNIX epoch time is the time elapsed since 19700101T00:00:00+0000 - number of leap
seconds added since 19700101T00:00:00+0000.

In principle this means that the epoch time repeats during the leap second.
In practice smearing is the better approach taken by many systems with the
clock remaining monotone, but briefly running more slowly (for added leap
seconds).

So you're not guaranteed precise synchronization with any particular clock, but
there is a deterministic conversion from epoch time to local time that does not
need to concern itself with leap seconds. This conversion will never produce
a 60th second, but could produce a 59th second that never existed if there
are ever negative leap seconds.

[Neil Mayhew]


 you can convert unix timestamps to some "local time without timezone
info" data structure (e.g. LocalTime); whether you assume UTC or any
timezone becomes relevant when you want to convert that data structure
into a data structure that does contain timezone info explicitly (like
ZonedTime) or implicitly (like UTCTime).

I don't think this is possible. If you want to measure seconds since
1970-01-01 00:00:00 local time (which may actually be 1969 in UTC) you
have to know the timezone at the start (ie in 1970) and at the end (ie
after the seconds have elapsed). That requires knowing whether DST
applies at either end, and whether the location was rezoned during the
intervening time. This requires IO and the conversion from seconds is no
longer a pure operation.

IHMO, the only sane way to interpret seconds-since-the-epoch is using UTC.

[Joachim Durchholz]

Post by Neil Mayhew

… you can convert unix timestamps to some "local time without timezone
info" data structure (e.g. LocalTime); whether you assume UTC or any
timezone becomes relevant when you want to convert that data structure
into a data structure that does contain timezone info explicitly (like
ZonedTime) or implicitly (like UTCTime).

I don't think this is possible. If you want to measure seconds since
1970-01-01 00:00:00 local time (which may actually be 1969 in UTC) you
have to know the timezone at the start (ie in 1970) and at the end (ie
after the seconds have elapsed). That requires knowing whether DST
applies at either end, and whether the location was rezoned during the
intervening time.

Actually you need only know the time offset at start and end of
interval. Any intervening changes cancel out.

You do need to know whether the location had its time zone changed
(which happened multiple times in some areas I believe).

Post by Neil Mayhew
This requires IO and the conversion from seconds is no
longer a pure operation.

Eek. IO just to handle timezone is a really bad idea.
Just use a timestamp-with-timezone data type for the parameters and
you're back with pure functions.

Post by Neil Mayhew
IHMO, the only sane way to interpret seconds-since-the-epoch is using UTC.

Now that's true. From the point of view in the paragraph above, simply
because it's lacking the data needed for any other interpretation.

Note that there are excellent designs for time representation arounds. A
good one would have to cover instants (multiple representations),
intervals (with and without leap seconds), time zones, and calendars.
From my daily Java work, the one that works best is the Jodatime
library, documented at http://www.joda.org/joda-time/. If anybody wishes
to check whether it has good ideas worth stealing, take a look at the
Javadoc at http://www.joda.org/joda-time/apidocs/index.html.

If you want to be really thorough, you can also study JSR-310, which was
supposed to be a slightly improve version of Jodatime, AND part of the
Java standard library. That work stopped when it was "good enough",
which isn't good enough. Still, the differences between Jodatime and
JSR-310 may be interesting in themselves (they may show areas where the
Jodatime author believed his work could be improved, as he was deeply
involved in the JSR as well).
Anyway: the JSR-310 classes are documented at
https://docs.oracle.com/javase/8/docs/api/index.html, in the java.time
package and subpackages.

Regards,
Jo

[Stefan Monnier]

Post by Tobias Dammers
It's even worse - unix time may also refer to the host's local timezone,

I don't know any unix-style system which does that.
Any detail on where/when this could happen?


Stefan