File size limit?

File size limit?

Posted by: creddick
Posted on: 2007-01-18 16:11:00

I have a large (2.73GB) quicktime file that I uploaded. In my FTP progam, the file completed the upload and i can download the entire file back to my machine locally with no problems. HOWEVER, when trying to access the file via the web, i receive a 403 Forbidden error with a 404 additionally. I'm clicking on the file name being shown in the 'directory listing', I know the file is there. Other smaller files function fine in the same folder. here's the folder:

http://www.quantizemedia.com/feifei/

what gives????

Re: File size limit?

Posted by: scjessey
Posted on: 2007-01-18 16:19:00

As I understand it, Linux is unable to handle a file size greater than 2 gigabytes.

Re: File size limit?

Posted by: creddick
Posted on: 2007-01-18 16:23:00

That sucks. I'll deal with it.

Re: File size limit?

Posted by: hcarty
Posted on: 2007-01-18 16:29:00

That 2gig limit is not correct - at the smallest, Dreamhost's filesystem should be able to handle 4gigabyte files. Realistically, file size limits are probably much larger than that.

The limitation is probably in Apache (the web server) or the browser used to download the file. It's probably worth emailing Dreamhost support over, as they would have a better idea of where the issue is.

Re: File size limit?

Posted by: scjessey
Posted on: 2007-01-18 16:42:00

In reply to:

That 2gig limit is not correct


I guess I didn't understand it then!

Re: File size limit?

Posted by: jasonj75
Posted on: 2007-01-18 18:28:00

I have a 29G file sitting on the server right now so the file system / OS isn't the issue...

Apache however is....v1.x won't serve files larger than 2G and v2.x is 4G (I'm not positive on the limit for 2.x, someone speak up if I'm off).

curl -I http://yoursite.com/ from the shell will tell you what version you are running on.



Re: File size limit?

Posted by: hcarty
Posted on: 2007-01-18 18:37:00

The filesystem limits have changed recently, and are different depending on the specific filesystem used - there's a lot of variation.

Re: File size limit?

Posted by: matttail
Posted on: 2007-01-18 18:48:00

Right, Apache is the issue here. I know the 1.x version can only handle up to 2 gig files, and I'm not sure about 2.x - which apparently is being used on newer servers and domains that have or had subversion installed.

Your best bet to work around this would be breaking up the file into several smaller chunks that can be downloaded and then reassembled - or host the file through a torrent network somewhere else. There's no reason you couldn't host the .torrent file here as well, you just can't install any sort of torrent software on the server.





--Matttail
art.googlies.net - personal website

Re: File size limit?

Posted by: vicm3
Posted on: 2007-01-18 19:15:00

For the note ext3, on a 32bits kernel:

Q: What is the largest possible size of an ext3 filesystem and of files on ext3?

inspired by Andreas Dilger, suggested by Christian Kujau:

Ext3 can support files up to 1TB. With a 2.4 kernel the filesystem size is limited by the maximal block device size, which is 2TB. In 2.6 the maximum (32-bit CPU) limit is of block devices is 16TB, but ext3 supports only up to 4TB.

From: http://batleth.sapienti-sat.org/projects/FAQs/ext3-faq.html

For what I know on a 64bits CPU, its posible to go to 16TB without a problem... (personally I don't have the right hardware to test :D), anyway if they were using XFS (not recomended if you dont had a very good UPS and a shutdown script to umount before a crash... ) the max file size is 8 exabytes...

just for the record

Re: File size limit?

Posted by: CiscoKid
Posted on: 2007-04-19 08:47:00

Apache 2.x also has the 2GB file limitation.I believe this was solved in the newest version 2.2.

-Ryan

Tags: 403 forbidden errordirectory listinglocallyuploadedquicktime