Hidden files exist because of a bug (probably)

Those files starting with . that appear when you run ls -a: why does a dot make a file hidden? The famous story is that this was actually a bug in early Unix. According to Rob Pike, who worked alongside the Unix authors, when . (current directory) and .. (parent directory) were added to directories, a quick check went into ls to keep them out of the listing: skip any name starting with .. That check accidentally hid every dot-prefixed file as well.

It was unintended, but it kept configuration files out of sight, so it stayed. Pike himself calls it a mistake that left home directories cluttered with config files. Even so, the convention that "files starting with . are hidden" became a Unix tradition, carried forward into macOS and Linux to this day. A bug became a feature, and a feature became a culture.

The history of software is built on happy accidents just like this one.