FTP uploads

Some questions for ftp upload:

How does a process check if a file upload is complete?

• Wikipedia:

(FTP has) no integrity check on the receiver side. If a transfer is interrupted, the receiver has no way to know if the received file is complete or not. Some servers support extensions to calculate for example a file's MD5 sum (e.g. using the SITE MD5 command) or CRC checksum, however even then the client has to make explicit use of them. In the absence of such extensions, integrity checks have to be managed externally.

• Solution: Pure upload script: ​http://linux.die.net/man/8/pure-uploadscript

     pure-uploadscript - Automatically run an external program after a successful upload
  
  

• Users that close connection and leave unfinished files behind, should solve this themselves. Solution could be to use ffmpeg to check video-file-validity.

User management connected to sponge users in db

• FTP daemon has to use external user/authentication management.

• PureFTP documentation: mysql or postgreSQL user management:

Here's a sample configuration file:



PGSQLServer     localhost

PGSQLPort       5432

PGSQLUser       root

PGSQLPassword   rootpw

PGSQLDatabase   pureftpd

PGSQLCrypt      cleartext

PGSQLGetPW      SELECT "Password" FROM "users" WHERE "User"='L'

PGSQLGetUID     SELECT "Uid" FROM "users" WHERE "User"='L'

PGSQLGetGID     SELECT "Gid" FROM "users" WHERE "User"='L'

PGSQLGetDir     SELECT "Dir" FROM "users" WHERE "User"='L'

• FTP daemon has to allow access only to private/home incoming dir.
  Covered by PureFTPd (chroot options).

Troubles while installing

• in /etc/pure-ftpd/conf are files with names specified in /usr/sbin/pure-ftpd-wrapper which will turn on/off certain command parameters of pure-ftpd
• MinUID will deny all users with lower uid's than in this file
• sometimes I had to kill all pure-ftpd processes to update my config changes:

  sudo killall -9 -r pure-.*
  
  

• postgresql settings are in /etc/pure-ftpd/db/postgresql.conf
• We need chroot to restrict users to their home dir: ChrootEveryone : yes
• Log can be seen through /var/log/syslog
• verbose: LogVerbose: yes

Upload script

The /etc/init.d/pure-ftpd script is broken and does not start the pure-upload script. So i edited it and added:

...

start_uploadscript() {

    UPLOADSCRIPT=/sponge/Sponge/python/sponge/storage/upload-complete.py # path to upload script!

    UPLOADUID=1000

    UPLOADGID=33

    if [ "$UPLOADSCRIPT" ... 

Upload script n