FTPS using Explicit TLS howto (Server)
First, you'll want to create a certificate, this can be used in the Certificate Generator in FileZilla Server. The Generator will want the country code, state, city, etc... Be as truthful as possible, you only undermine your own credibility if you enter wrong information into the certificate.
The key size for the certificate is chosen at the top of the generator: 1280 bit, 2048 bit, 4096 bit. The bigger the key size the more secure the certificate and the initial session key exchange on every connection will be. There is however one thing that needs to be taken into account, CPU utilization during the connection handshake. When you apply encryption to your FileZilla server the CPU will have to do many calculations to encrypt the data being sent and decrypt the data being received. Bandwidth will also play a factor in how much the CPU is being utilized. If you have a slower connection, let's say around 1.5Mbps up you may not have to worry about CPU utilization as much. The best way to decide is to test.
Please note that FZS needs the paths to the certificate files: If you generate your own private key and certificate without putting a path in front of the file name, FZS only puts the bare filename in the certificate field without an error notice, but later you will get "Could not load certificate file" errors in the FZS log when someone tries to connect via FTPS/FTPES (Implicit/Explicit).
Therefore always put the full path to the private key and certificate files in their corresponding fields and FZS can find the files.
If your server has a direct connection to the internet the configuration is simple, check "Enable FTP over TLS support (FTPS)".
More FTPS documentation is available here.
Configure with NAT
Please read the Network Configuration guide for instructions on how to configure the server behind NAT devices (Router, Firewall, etc).
Enable Explicit FTP over TLS
On the TLS settings page check "allow Explicit FTP over TLS." It is recommended to also check "Disallow plain unencrypted FTP" and "Force PROT P to encrypt file transfers when using FTP over TLS". This will further enforce encryption policies; here PROT "P" is for "Private" as opposed to "C" for clear text. If you only want certain groups or users to have encryption you can set that up in the user or group editor. If there is data you still want available to the general public the "Force" setting should be disabled in the server settings menu, as you will need an FTP client rather than a web browser to access the FTP server. If using "PROT P - Private", the client may require a matching TLS setting or it may default to PROT C.
Another option you should enable is "Require TLS session resumption on data connection when using PROTP P" as it protects against data connection theft.
Setting up your FTP server in this way allows you to encrypt your data and login information without having to get 3rd party programs. With explicit TLS you will need an FTP client. Internet Explorer and Firefox don't support TLS without special plugins. FileZilla client supports FTPS both implicit (FTPS:// protocol), and explicit (FTPES://).