Bash, or the Bourne-Again SHell, is a CLI that was created in 1989 by Brian Fox as a free software replacement for the Bourne Shell. A shell is a specific kind of CLI. Bash is open source. In most ways Zsh is an exact replacement for Bash, so there is no need to switch over or install Bash instead.
When using the command line, we refer to folders as directories. Files and directories on your computer are organized into a filesystem.