If you look at the age of the universe vs. how long we expect the universe to last, we are in the super early beginnings. There are supposedly countless trillions of years until the heat death of the universe.
What if we're just early to the party? Obviously earth is pretty unique compared to most other planetary bodies we have identified. So not only do you need a bit of time, but also a bit of luck to meet all the criteria for life as humans know it to exist. Not saying their aren't other ways life can evolve, but for carbon based life the window seems pretty small.
I also wonder about what an advanced civilization would do to mask it's presence for it's own safety. I imagine the concealment of an advanced civilization would be so similar to nature that to the untrained observer it would look just like any other system or galaxy. Unless you were so sure of your superiority that you wanted others to find you, I would guess hiding in plain sight would be the safest option.
I do find it puzzling that we haven't seen anything else out there, but until a couple dozen centuries ago we thought the gods controlled the rising and setting sun, so maybe we don't have the right technology yet. Imagine trying to explain WiFi to Plato. Colorless, tasteless, odorless, signals passing information all around us.
Then you have things like Hoag's Object and maybe they are just messing with us.