You might wonder if this time problem is related to Y2K (which also had a similar name, "year 2000 problem"), which was a big worry going into the new millennium. While both revolve around computers keeping time, the past problem was caused by decimal shorthand, not binary overflow.
In the early days of computing, memory was extremely expensive, so programmers saved wherever possible. It was thus common to use two digits to indicate the year, assuming that the first two digits were always "19". Y2K was the worry that once the year hit 2000, computers would interpret "00" as 1900 (or another earlier date) instead of 2000, causing all kinds of problems for programs that relied on accurate time. It was a big scare at the time, but ended up not being nearly as terrible as expected (which, depending on who you ask, was either due to prevention efforts or because the issue was never huge to begin with).
Other time-tracking methods
Unix time is not the only way that computers handle the clock. Many machines have a real-time clock, which keeps track of the time similarly to a wall clock and retains power using the CMOS battery when the computer is powered off. When this battery dies, your computer's clock won't be right.
Windows, as the exception mentioned earlier, keeps track of time differences in 100-nanosecond ticks starting January 1, 1601 at midnight UTC. While it's much older than 1970, this epoch isn't random; the Gregorian calendar we use goes through a complete cycle every 400 years, so 1601 was the start of the most recent cycle at the time of Windows' development. It's an unsigned 64-bit integer value (as it's unlikely a computer would need to represent dates before 1601) that will become invalid on September 14, 30,828.
Finally, there's NTP (Network Time Protocol), which syncs your computer's time to extremely accurate time-keeping servers over the internet. If the computer's method of keeping the time ever fails or drifts, it receives corrections using NTP. There's a lot more to keeping track of time, like soon-to-be-retired leap seconds, daylight saving time, time zone changes, and calendar shifts, but now you understand more about what computers do to keep track of it.