Failed to establish connection to ftp.ebi.ac.uk

Data Access Data downloads
1

bug:
Hello, I have problems accessing the datasets on ftp.ebi.ac.uk via FTP (port 21).

Current behaviour:
I’m facing below issues
Error1:“425 Failed to establish connection".
Error2: “421 service not available remote service has closed”.

Expected behaviour
I expect to connect and download the datasets via FTP as it worked for me before.

Could you please help us on how to download the datasets through ftp connection.

Hostname: ftp.ebi.ac.uk

Datasets to be downloaded:
1.Opentargets: /pub/databases/opentargets/genetics/latest/
2.Opentargets non genetics: /pub/databases/opentargets/platform/latest/
2.VGNC: /pub/databases/genenames/vgnc/
3.HGNC: /pub/databases/genenames/hgnc/

2

Hi @Vijay,

The EBI FTP server does have intermittent problems sometimes, so first of all I suggest that you check again if the problem has been resolved by now.

If that doesn’t work, could you share your OS (Linux/Windows/Mac) and which software are you using to fetch the files from the FTP?

3

@Kirill_Tsukanov From the past two weeks we are facing the issue . We tested again today but still facing the same issue.
Here are the OS details.
NAME=“Red Hat Enterprise Linux”
VERSION=“8.7 (Ootpa)”
ID=“rhel”
ID_LIKE=“fedora”
VERSION_ID=“8.7”
PLATFORM_ID=“platform:el8”

We are using FTPClient in linux for fetching files.

Could you please help us on how to download the files. Let us know if you need any further details.

4

Hi @Vijay,

Can you try downloading with wget and see if it works for you? For example:

wget -m ftp://ftp.ebi.ac.uk/pub/databases/opentargets/genetics/latest/

If it still doesn’t work, please send the error message the above command shows you.

1 Like
5

Hello @Kirill_Tsukanov
We ran the below command
Command: wget -m ftp://ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/ &> log_test.log
There are a total of 31404 files in the above mentioned folder but we can see that only 12000 files are downloaded using the above given command, remaining all files are failed with the error:

2023-10-23 23:01:34 (0.00 B/s) - Data connection: Connection timed out; Retrying.

--2023-10-23 23:01:35--  ftp://ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/json/errors/evidence/sourceId%3Dimpc/part-00037-91841997-85c5-4805-89bc-636bcba235ae.c000.json
  (try: 2) => ‘ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/json/errors/evidence/sourceId=impc/part-00037-91841997-85c5-4805-89bc-636bcba235ae.c000.json’
==> CWD not required.
==> PASV ...
Cannot initiate PASV transfer.
--2023-10-23 23:01:35--  ftp://ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/json/errors/evidence/sourceId%3Dimpc/part-00038-91841997-85c5-4805-89bc-636bcba235ae.c000.json
           => ‘ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/json/errors/evidence/sourceId=impc/part-00038-91841997-85c5-4805-89bc-636bcba235ae.c000.json’
==> CWD not required.
==> PASV ...
Error in server response, closing control connection.
Retrying.
1 Like
6

@Kirill_Tsukanov
Can you please look into the previous comment.

1 Like
7

Hi @Vijay,

Thank you for posting the error message.

Looking at the command you’re using, you don’t need to fetch the entire etl/ directory, because its subdirectories parquet/ and json/ contain exactly the same data, just in different formats. So as the first suggestion, try downloading just one of the subdirectories, depending on which format you want. Perhaps reducing the number of files will help with the reliability of the transfer.

Another flag which sometimes help inrease reliability is --passive-ftp. This way, wget needs to open less connections for a download, also potentially increasing stability.

Taking the above suggestions into account, then, this is the command that I suggest you use (assuming you want “json” format; substitute it with “parquet” otherwise):

wget -m --passive-ftp ftp://ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/json/ &> log_test.log

Please let me know if the same (or different) error occurs, and I’ll look into it further.

1 Like
8

@Kirill_Tsukanov
We ran the command that is provided in the previous comment.
Command:
wget -m --passive-ftp ftp://ftp.ebi.ac.uk/pub/databases/opentargets/platform/latest/output/etl/json/ &> log_test1.log

The job is still running since 14 days and 13298/15703 files succeedded which usually takes one day approximately for the job to complete all the files.

1 Like
9

Hi @Vijay, I’m sorry to hear you are still having issues.

At this point I think we’ve tried most options we could. The FTP server is managed by EBI directly, rather than by Open Targets, so as your next step I suggest you contact EBI support: Support and feedback | EMBL-EBI — please explain your problem and the issues you had trying to download the data. (You can send a link to this post to avoid repeating all steps that we tried and the error messages.)

1 Like
10

Sure @Kirill_Tsukanov will contact the EBI support team.

Thank you for helping us.

1 Like
11

@Kirill_Tsukanov The EBI team is asking to raise ServiceNow ticket to technical team using below link.
link:Missing cookie

But we don’t have any account with the mentioned ServiceNow portal.
Please find the below screenshot of login page.

image
image886×823 26.3 KB

We can’t find any create account option in the above login page.
Can you please help us in proceeding further.

1 Like
12

@Kirill_Tsukanov can you please help us in connecting with the EBI team.
We are facing the issue which is mentioned in the previous comment.

1 Like
13

Hi @Vijay, I suggest that you reply to the EBI support and point out that you are not EBI staff and so do not have a ServiceNow account.

Unfortunately I cannot raise this issue from my side, because I’m not able to reproduce this specific error.

2 Likes
14

Hi @Vijay!

I think this is the page you need to use if you are an external user:

Best wishes,

Helena