I've done this plenty of times.
I could do this with a Debian (like with DSL or Knoppix), Gentoo or Sabayon (another Gentoo derivative), Slackware, or LFS userland. If you also need the executables really small, then I also could build a custom busybox env to get the footprint small like for various embedded environments.