Rootless containers are mostly a non-event now, which is the highest praise I can give a feature. Podman runs my workloads as my own user, no daemon owning everything as root, and for the common case it just works. But there are about four things that bite the first time, so here they are before you go looking.
First, the subordinate id ranges. Rootless needs entries in /etc/subuid and /etc/subgid to map container users into your namespace. If a container starts but files have weird ownership or a build fails halfway, check those first.
grep "$USER" /etc/subuid /etc/subgid
podman system migrate # after changing the ranges
Second, low ports. You can't bind below 1024 as an ordinary user, so your container's 80 maps to a high host port and you reverse-proxy to it, or you set net.ipv4.ip_unprivileged_port_start. I prefer the proxy.
Third, services that outlive your login. A rootless service stops when your session ends unless you enable lingering, which is the one that catches everyone deploying a long-running container.
loginctl enable-linger "$USER"
Generate a systemd unit with podman generate systemd, enable it as a user service, and it survives reboots and logouts. After that it genuinely is set-and-forget, and I sleep better knowing nothing here is running as root.