Editing Storj

[[Category:Documentation]]
Storj is a decentralized peer-to-peer file storage system, and new protocol being added to FileZilla. 

Source code is available at https://svn.filezilla-project.org/filezilla/FileZilla3/trunk and depends on https://github.com/Storj/libstorj

To checkout the code with subversion:

    svn co https://svn.filezilla-project.org/svn/FileZilla3/trunk filezilla

And enable using:

    ./configure --enable-storj

You can download the current release candidate that includes Storj from: https://filezilla-project.org/download.php?type=client&beta=1


= Quick setup =

* Add a new site in the Site Manager and select Storj as the protocol
* Enter '''api.storj.io''' as the host and '''443''' as the port
* Enter the bridge user name and password
* If you have not created an Encryption Key before, generate a new one, otherwise add your existing key
* Connect to the server and begin uploading and downloading files


= 1. Introduction = 

[https://filezilla-project.org/ FileZilla] is a fast and reliable cross-platform FTP, FTPS, SFTP and Storj client with many useful features and an intuitive graphical user interface. A server client is also available. 

Below are some of the FileZilla client features:

* Easy to use
* Supports Storj, FTP, FTP over SSL/TLS (FTPS) and SSH File Transfer Protocol (SFTP)
* Cross-platform. Runs on Windows, Linux, *BSD, Mac OS X and more
* IPv6 support
* Available in many languages
* Supports resume and transfer of large files >4GB
* Tabbed user interface
* Powerful Site Manager and transfer queue
* Bookmarks
* Drag & drop support
* Configurable transfer speed limits
* Filename filters
* Directory comparison
* Network configuration wizard
* Remote file editing
* Keep-alive
* HTTP/1.1, SOCKS5 and FTP-Proxy support
* Logging to file
* Synchronized directory browsing
* Remote file search

FileZilla incorporates the Storj asynchronous multi-platform C library for encrypted file transfer on the Storj network into its back end to ensure the highest level of performance and stability. This library is also the back end to the native [https://docs.storj.io/v1.1/docs/getting-started libstorj CLI].

In this guide, we will run through the process of setting up FileZilla to be able to up and download painlessly to the Storj Cloud.

= 2. Client Installation = 

A FileZilla installation guide for your specific operating system can be found [https://wiki.filezilla-project.org/Client_Installation on the official FileZilla Wiki page].

After the installation is complete, exit the installation window and launch the client.


= 3. Setting up a Storj account  = 

Before we can start uploading and downloading to the Storj network, it is necessary to open an account on the Storj Bridge. The Bridge acts as an interface to the Storj network and manages all the complicated stuff in the background, making it completely painless for a user to start using the Storj network. Please note that the Bridge maintained by Storj Labs is not the only bridge the user can connect to, it is also possible to set up a Storj network in a contained environment using the [https://github.com/Storj/storj-sdk Storj Development Kit]. Also, other bridges exist on the network (e.g. for testing) to which one may choose to connect.

'''Note:''' If you want to register to a bridge that does not have a graphical user interface, you will have to register via [https://docs.storj.io/v1.1/docs/getting-started libstorj CLI] first and then import the keys into FileZilla.


== 3.1. Registering an account via Bridge-GUI ([https://app.storj.io app.storj.io]) ==

It is possible to register directly via bridge-GUI by clicking on [https://app.storj.io app.storj.io] or pasting <code>app.storj.io</code> into a web-browser (Figure 3.1). If you already have a Storj account, you can proceed directly to section 3.2.


[[File:Storj bridge register GUI.png|300pix|thumb|center|Figure 3.1. Bridge-GUI (app.storj.io) main Login window, click on "Sign Up".]]


To register a new account, click on the [https://app.storj.io/#/signup Sign Up link] (Figure 3.1). Once on the registration page, enter your email address and desired password, then click the check box to agree to the terms of service and finally click on "Sign Up" (Figure 3.2).


[[File:Storj sign-up.png|300pix|thumb|center|Figure 3.2. Bridge-GUI (app.storj.io) main registration page.]]


Now check your email inbox and click on the account activation link sent to you by Storj. A web page should open with the account activation confirmation. Note that sometimes the email automatically gets moved to the spam/trash folder by some email providers, so if the email is not in the inbox, make sure to check the spam folder. Now
that the bridge account is created, you can login to bridge-GUI (Figures 3.1) and you will be presented with your Bridge-GUI dashboard page as show in Figure 3.3.


[[File:Storj app.storj.io login page.png|400pix|thumb|center|upright=2.0|Figure 3.3. Bridge-GUI (app.storj.io) dashboard.]]


== 3.2. Configure billing to upload/download beyond the free tier limit ==

Currently the free tier for Storj renters is structured as follows:

*   '''Uploading limit''': 3GB/hour, 10GB/day and 25GB/month.
*   '''Downloading limit''': 9GB/hour, 30GB/day and 180GB/month.

(Data obtained from: [https://github.com/braydonf/storj-bridge/blob/59a04626628578a5196034ae0825f5eb13661e43/lib/config.js#L60-L69 storj-bridge]. If you want to upload or download beyond the free tier, a payment option can be added to the Billing tab of your account on [https://app.storj.io app.storj.io]. 


'''Payments options currently include:'''

* STORJ token (this currently needs to be requested by email to '''hello@storj.io''', automatic deposit option will be added later to the GUI)

* BTC

* Credit Card


[[File:Storj billing.png|400pix|thumb|center|upright=2.0|Figure 3.4. Adding a payment option to 'Billing' allows you to upload and download beyond the free tier limit.]]


Once your billing information is added, the free tier limit will be lifted and you will now be able to upload and download as much as you like/can afford!

''These are the current storage and bandwidth prices ('''they apply only after you exhausted your monthly free tier limits'''):''

*   '''Storage''' -> $0.015 per GB per month
*   '''Download Bandwidth''' -> $0.05 per GB downloaded
*   '''Upload bandwidth''' -> free

This information was extracted from [https://storj.io/pricing.html pricing].


'''Important things to note regarding the billing structure:'''

* You only pay for the time span that your data was stored on the network, if you delete some of your files today you will not pay for them anymore tomorrow, '''you will stop getting charged for storage right after you deletes the files'''.
* You storage usage is added daily to your ''"Current Usage"'' in the billing section. However, if your billing method is by credit card, you will only be billed once per month (normally on the 22nd of every month). For pre-paid billing methods like STORJ token or BTC, your pre-paid balance will be reduced daily according to your usage but again actual amount deducted from your balance will be reconciled at the end of the billing cycle taking into account your free tier allotment.
* '''The free tier allotment is subtracted from your paid tier every month''', thus if you upload 100GB in a given month, you will only pay for 75GB (100GB – 25GB free tier) at the end of the billing cycle.

= 4. Configure FileZilla so it can be used with Storj = 


After the main FileZilla client window opens do the following:

* Click on <code>File</code> -> <code>Site Manager...</code>


[[File:Storj Site Manager 1.png|400pix|thumb|center|upright=2.0|Figure 4.1. Go to <code>File</code> -> <code>Site Manager...</code>]]
	

* In the resulting Site Manager Popup Window, first click on the <code>New Site</code> button at the bottom left-hand side. You can <code>Rename</code> the site name to a name of your liking, for example <code>Storj</code>. Then in the <code>General</code> tab on the right side, for <code>Host:</code> enter <code>api.storj.io</code>, for <code>Port:</code> enter <code>443</code> and for <code>Protocol:</code> select <code>Storj - Decentralized Cloud Storage</code>.


[[File:Storj Site Manager 3.png|400pix|thumb|center|upright=2.0|Figure 4.2. Site Manager menu.]]

	
* In the <code>User</code> field enter your [https://api.storj.io/ api.storj.io] '''email address'''. In the <code>Password</code> field '''enter the matching password to the previous email address'''.
** '''If you are new to Storj''' click on <code>Generate...</code> to the right of the <code>Encryption key</code> field. This will generate an encryption key which will be used to encrypt all the data you upload to Storj, including all the folders. This key will also be used to decrypt the data you download from your Storj account. '''Be sure to write down the encryption key in a very safe place as you will lose access to your uploaded data if you don´t have the encryption key.'''
** '''If you were previously running libstorj''' see the next chapter to export the encryption key from libstorj. Once the encryption key is exported, you can paste it into the <code>Encryption key</code> field in FileZilla. This way you will be able to access any files you previously uploaded to Storj with libstorj. The same applies if you were using core-CLI or any other toolkit to upload to Storj.
* On the bottom right, there is a space to add <code>Comments</code> to the account which is particularly handy if you want to add more accounts to FileZilla.
* Finally, click on <code>Connect</code>. FileZilla Should now connect to your account. If all goes well and the login details are correct, no errors will be returned and the Site Manager will close (Figure 4.3).


'''Note:''' If already added any buckets (folders in FileZilla) to you account with another application, you will get the following error:

    Error: Wrong encryption key for at least one bucket

You do not have to worry about this error, it just means you cannot access one of the folders because you do not have a valid decryption key for it. In certain use cases this is actually desirable because sometimes you want to keep the file encryption key private for a certain machine/environment.

[[File:Storj Main dashboard.png|400pix|thumb|center|upright=2.0|Figure 4.3. Dashboard after successfully logging in to our Storj account.]]


'''Note:''' It is absolutely vital that you backup the encryption key to a safe location and preferentially offline. Printing the key on paper is a very secure common practice.


== 4.1 Export encryption key from libstorj ==

If you were previously running libstorj, you can export the encryption key by executing the following command (Figure 4.4):

* <code>storj export-keys</code>


[[File:Storj export-keys.png|400pix|thumb|center|upright=2.0|Figure 4.4. Export the encryption key from libstorj for use in FileZilla.]]


Now copy the <code>Encryption key</code> string exported from libstorj and paste it into the <code>Encryption Key</code> field in the FileZilla Site Manager menu.

== 4.2. Secure the FileZilla keys with a master password ==

To protect the encryption keys and bridge password, it is advisable to encrypt these passwords with a top-layer master password (and/or use full disk encryption and home directory encryption). This can be done as follows: 

#. In the FileZilla top menu, click on <code>Edit</code> -> <code>Settings...</code>
#. Once in the <code>Settings</code> menu, click on <code>Interface</code>.
#. Under <code>Passwords</code> select <code>Save passwords protected by a master password</code>.
#. Now enter a password of your choice (preferentially a long one), repeat the password for confirmation and click "OK".


[[File:Storj password for seed.png|400pix|thumb|center|upright=2.0|Figure 4.5. Configure a master password to protect your bridge password and encryption keys]]

Done!. Now in case your computer gets compromised, it will be impossible for the hacker to retrieve your bridge keys and encryption key without using a key logger while you're unlocking the password.

== 4.3 Adjust simultaneous transfers and timeout details ==

We can adjust the number of simultaneous transfers to a value from 1 to 10. Setting this value to 10 allows maximum bandwidth utilization during transfers of a large number of files, both for uploading and downloading.

This can be done as follows:

# In the FileZilla top menu, click on <code>Edit</code> -> <code>Settings...</code>.
# Once in the <code>Settings</code> menu, click on <code>Transfers</code>.
# In the <code>Transfers</code> section set the <code>Maximum simultaneous transfers</code> to a number of your choice between 0 and 10.


[[File:Storj settings 2.png|400pix|thumb|center|upright=2.0|Figure 4.6. Configure the number of maximum simultaneous transfers for optimal bandwidth utilization.]]


'''Note:''' You can also set transfer speed limits in KiB/s for both downloads and uploads in this settings menu. However, this isn't currently available with Storj, please watch https://github.com/Storj/libstorj/issues/9 for when it's added.

Timeouts can occur at the default timeout value. We can adjust this value upwards to where timeout errors no longer occur. In the example below the timeout was set to 60 seconds. This will affect all commands except for uploads and downloads.

[[File:Storj settings 1.png|400pix|thumb|center|upright=2.0|Figure 4.7. Adjust timeout limit in seconds.]]


'''Note:''' The <code>Maximum number of retries</code> is also a useful setting to adjust in case downloads or uploads fail after the first 1-2 retries.

= 5. Creating a folder, uploading and downloading files =

This step is very easy. Once connected to our account, there are two main sections of the GUI, namely:

* Local site (your computer) - Left pane
* Remote site (the Storj cloud) - Right pane

If you already have some folders (buckets) in your Storj account, they will be displayed in the <code>Remote site</code> pane. You can easily create a folder by right clicking somewhere in the remote site pane and selecting <code>Create directory</code> (Figure 5.1).

'''Note:''' You can add as many folders as you want and also add sub-folders!.


[[File:Storj Add a folder.png|400pix|thumb|center|upright=2.0|Figure 5.1. Adding folders to Storj for uploading files.]]


You can now browse to the files/folders you want to upload in the <code>Local site</code> pane and upload them by dragging-and-dropping them to the Storj folder you created in the <code>Remote site</code> pane. 


[[File:Storj upload.png|400pix|thumb|center|upright=2.0|Figure 5.2. Uploading files to a folder/bucket.]]


'''Tip:''' If you can't add a directory, you can reconnect to Storj by doing the following:

* Go to <code>Server</code> -> <code>Reconnect</code> (Figure 5.3).


[[File:Storj reconnect to server.png|400pix|thumb|center|upright=2.0|Figure 5.3. Reconnecting to Storj.]]


Downloading files is just as easy, only this time drag-and-drop the files from your folder in your <code>Remote Site</code> pane to a folder of your choice in the <code>Local Site</code> pane. That's it!


= 6. Frequently Asked Questions =

=== What is an Encryption Key? ===

The Encryption Key is a series of 12 to 24 words that represents 128 to 256 bits of entropy that is used to encrypt files before they are uploaded into the network. 

Here is an example Encryption Key:

    legal winner thank year wave sausage worth useful legal winner thank year wave sausage worth useful legal will

If you're familiar with BTC, this will be recognizable as it's based on the [https://github.com/bitcoin/bips/blob/master/bip-0039.mediawiki BIP39] standard.

=== How do I export settings from libstorj CLI? ===

First run this command to get the bridge user, password and the encryption key:

    storj export-keys

And then enter those settings into FileZilla in the Getting Started section.

=== What does it mean if I see "Wrong encryption key for at least one bucket"? ===

This means that a bucket was added using a different Encryption Key and you won't be able to download the files within that bucket as it won't be able to decrypt or verify their integrity.

=== Why is there more data transfered over the wire compared to the total size of the file? ===

Files are encoded with [https://en.wikipedia.org/wiki/Reed%E2%80%93Solomon_error_correction Reed Solomon] error correction before they are uploaded to the network. This provides the ability to repair parts of the file that are missing or inaccessible. The number of parity shards is currently calculated as 2/3 of the original file size.

=== How are files encoded before being uploaded to the Storj network? ===

Files are first encrypted using the Encryption Key and then encoded using Reed Solomon into various size shards. For specific details please read [https://github.com/Storj/sips/blob/master/sip-0005.md SIP5]. These shards are then uploaded to different nodes in the Storj network directly after communicating with the Storj bridge to negotiate which nodes to communicate with. After the file is uploaded, it is then mirrored to other nodes in the network, so that in the event that a shard goes missing after this initial upload, a shard can be repaired. The total number of shards that can be repaired is around 1/2 of the original file. 

'''''Note:''' For larger files the time it takes to do this preparation step can be more noticeable.''

=== Can I rename or move files? ===

Not currently, this feature can be supported in the future, when file ids are unique, see deprecation messages at https://github.com/Storj/service-storage-models/blob/master/lib/models/bucketentry.js#L115-L119

=== What do I do if I see "Transfer rate limit" error? ===

This means that a rate limit on the number of bytes uploaded or downloaded has been reached. These are calculated time spans by hour, day and month. Accounts that have billing information entered currently do not have these rate limits, and instead have a rate limit on the total number of requests, which can be seen as a "Too many requests" error.

=== How many files can I upload? ===

There currently isn't a hard limit an the total number of bucket entries that can be added to a bucket. However there is limited support for more than 5000 buckets and more than 2000 bucket entries.

=== Why can't I upload a file at the root? ===

FileZilla currently maps buckets as directories at the root level and each file within those buckets is a bucket entry. Sub-directories are created by having a forward slash in the bucket entry name, which are parsed and shown as a directory within FileZilla. For this reason, it's not possible to add a file to the root because there isn't a bucket to add a bucket entry.


= 7. Conclusion = 

In this simple guide, we first created a Storj account and added a billing option. We then imported our account details into FileZilla and adjusted a few parameters in the client settings. Finally, we learned how to add a folder to out Storj account and how to up- and download files to and from this folder. FileZilla has many more advanced features and settings, so if you want to learn more about FileZilla, please have a look at the [https://wiki.filezilla-project.org official FileZilla wiki page].

We are constantly improving Storj and all of its tools to make sure that both renters and farmers can interact harmoniously in the Storj ecosystem. We would love to hear from you, no matter if you have suggestions, comments, questions or criticisms you would like to share. 

* Community chat: [https://storj.io/community.html Storj Community].
* Storj feedback: [https://wantoo.io/storj-product-feedback/ Storj product feedback].


= Community & Support =

Please sign up for the Storj community chat at: https://community.storj.io/ and ask any questions you may have in the #filezilla channel.

= Debugging =

It's possible to view full detailed logs for file transfers. To enable it ''(you'll need to do this each time you start up)'':

* Open up the Preferences/Settings
* Go to Logging and optionally enable timestamps and select a file to log
* Go to Debug and select "4 - Debug"
* View logs in the command console