[haiku-bugs] Re: [Haiku] #8473: Concerns about ISO 9660 implementation
- From: "umccullough" <trac@xxxxxxxxxxxx>
- Date: Sun, 15 Apr 2012 17:29:48 -0000
#8473: Concerns about ISO 9660 implementation
-------------------------------------+-----------------------
Reporter: scdbackup | Owner: nobody
Type: enhancement | Status: new
Priority: normal | Milestone: R1
Component: File Systems/ISO 9660 | Version: R1/alpha3
Resolution: | Keywords:
Blocked By: | Blocking:
Has a Patch: 0 | Platform: All
-------------------------------------+-----------------------
Old description:
> On occasion of ticket #8460 i noticed two potential problems of the
> ISO 9660 filesystem read-only implementation:
>
> It looks like data files of 4 GiB or larger are not supported.
> Even if it is not intended to augment Haiku for reading large data files,
> it should at least be tested whether it tolerates such files gracefully.
>
> It has not been challenged whether the block-to-byte address computations
> of Directory Entry addresses are safe against byte address of 4 GiB or
> larger.
> Only the address computations for data file content have been fixed by
> #8450.
>
> I am not a user of Haiku, but a producer of ISO 9660 filesystems.
> In that role i would be glad to help with better readability of ISO 9660.
>
> ------------------------------------------------------------------------
>
> Large data files:
>
> {{{struct iso9660_inode}}} in
> http://cgit.haiku-os.org/haiku/tree/src/add-
> ons/kernel/file_systems/iso9660/iso9660.h
> only has provisions for memorizing a single data extent:
> {{{
> uint32 startLBN[2]; // Logical block
> # of start of file/directory data
> uint32 dataLen[2]; // Length
> of file section in bytes
> }}}
>
> ECMA-119 (the public specification of ISO 9660) says:
>
> 6.5.1 Relation to File Sections
> "Each file shall consist of one or more File Sections. Each File Section
> of a file shall be identified by a record in the same directory. The
> sequence of the File Sections of a file shall be identified by the order
> of the corresponding records in the directory.
> A File Section may be part of more than one file and may occur more than
> once in the same file. A File Section may be identified by more than one
> record in the same or a different directory."
>
> So if {{{struct iso9660_inode}}} is indeed the 1:1 representation of a
> filesystem inode, then it should be in a 1:n relation to ISO 9660
> Directory
> Entries.
> (The array size [2] is not a 1:2 relation but rather mirroring the fact
> that ISO 9660 Directory Entries bear address and length in both byte
> orderings.)
>
> More challenging will be to collect all ISO 9660 Directory Entries which
> bear identical names in the same directory.
>
> But as said, at least it should be made sure that two ISO 9660 Directory
> Entries with the same name do not confuse Haiku beyond the scope of that
> single data file.
>
> Proposal for producing a challenging ISO image:
>
> Create a file with at least 4 GiB of size. Plus some small files with
> names alphabetically lower and higher than the big one's ("aaa", "zzz").
>
> Put them into an ISO 9660 image e.g. by my program
> {{{
> xorriso -outdev my_image.iso -add aaa file_of_4gb zzz --
> }}}
> or use mkisofs
> {{{
> mkisofs -o my_image.iso -R -iso-level 3 aaa file_of_4gb zzz
> }}}
> (xorriso can operate directly on DVD burners on systems with Linux
> kernel,
> FreeBSD, or Solaris. Like -outdev /dev/sr0 , -outdev /dev/cd0, ...)
>
> Mount the image and check whether all three files behave nice.
> E.g. whether file_of_4gb does not appear twice in a directory listing,
> whether at least the small files show correct content, ...
>
> ------------------------------------------------------------------------
>
> Directory entries in high block numbers:
>
> Normally the Directory Entries are stored in a few hundred blocks
> shortly after block 16. So there occur no large byte addresses.
> But in the case of multi-session, the new directory tree gets written
> after the data blocks of the previous session.
>
> So if the first session is 4 GiB or larger, then the address computations
> for ISO 9660 Directory can produce byte addresses which may get spoiled
> by 32-bit bottlenecks like the one of #8460.
>
> Proposal for producing a challenging ISO image:
>
> Produce an ISO image which contains two data files of exactly 2 GiB each:
> {{{
> xorriso -outdev my_image.iso -add file1 file2 --
> }}}
> (Note that other than mkisofs -o, this will refuse to overwrite an
> existing
> image.)
>
> Add a session to the image by adding a small marker file
> "this_is_session_2"
> {{{
> touch this_is_session_2
> xorriso -dev my_image.iso -add this_is_session_2 --
> }}}
> (Note that the second run uses option -dev rather than -outdev. By this
> it
> loads the tree of the first session and adds the new file to it.)
>
> The resulting file "my_image.iso" will bear two sessions. The PVD at
> block 16 will direct the reader to a root directory above 4 GiB.
> The image will still fit on a single-layer DVD.
New description:
On occasion of ticket #8460 i noticed two potential problems of the
ISO 9660 filesystem read-only implementation:
It looks like data files of 4 GiB or larger are not supported.
Even if it is not intended to augment Haiku for reading large data files,
it should at least be tested whether it tolerates such files gracefully.
It has not been challenged whether the block-to-byte address computations
of Directory Entry addresses are safe against byte address of 4 GiB or
larger.
Only the address computations for data file content have been fixed by
#8460.
I am not a user of Haiku, but a producer of ISO 9660 filesystems.
In that role i would be glad to help with better readability of ISO 9660.
------------------------------------------------------------------------
Large data files:
{{{struct iso9660_inode}}} in
http://cgit.haiku-os.org/haiku/tree/src/add-
ons/kernel/file_systems/iso9660/iso9660.h
only has provisions for memorizing a single data extent:
{{{
uint32 startLBN[2]; // Logical block #
of start of file/directory data
uint32 dataLen[2]; // Length
of file section in bytes
}}}
ECMA-119 (the public specification of ISO 9660) says:
6.5.1 Relation to File Sections
"Each file shall consist of one or more File Sections. Each File Section
of a file shall be identified by a record in the same directory. The
sequence of the File Sections of a file shall be identified by the order
of the corresponding records in the directory.
A File Section may be part of more than one file and may occur more than
once in the same file. A File Section may be identified by more than one
record in the same or a different directory."
So if {{{struct iso9660_inode}}} is indeed the 1:1 representation of a
filesystem inode, then it should be in a 1:n relation to ISO 9660
Directory
Entries.
(The array size [2] is not a 1:2 relation but rather mirroring the fact
that ISO 9660 Directory Entries bear address and length in both byte
orderings.)
More challenging will be to collect all ISO 9660 Directory Entries which
bear identical names in the same directory.
But as said, at least it should be made sure that two ISO 9660 Directory
Entries with the same name do not confuse Haiku beyond the scope of that
single data file.
Proposal for producing a challenging ISO image:
Create a file with at least 4 GiB of size. Plus some small files with
names alphabetically lower and higher than the big one's ("aaa", "zzz").
Put them into an ISO 9660 image e.g. by my program
{{{
xorriso -outdev my_image.iso -add aaa file_of_4gb zzz --
}}}
or use mkisofs
{{{
mkisofs -o my_image.iso -R -iso-level 3 aaa file_of_4gb zzz
}}}
(xorriso can operate directly on DVD burners on systems with Linux kernel,
FreeBSD, or Solaris. Like -outdev /dev/sr0 , -outdev /dev/cd0, ...)
Mount the image and check whether all three files behave nice.
E.g. whether file_of_4gb does not appear twice in a directory listing,
whether at least the small files show correct content, ...
------------------------------------------------------------------------
Directory entries in high block numbers:
Normally the Directory Entries are stored in a few hundred blocks
shortly after block 16. So there occur no large byte addresses.
But in the case of multi-session, the new directory tree gets written
after the data blocks of the previous session.
So if the first session is 4 GiB or larger, then the address computations
for ISO 9660 Directory can produce byte addresses which may get spoiled
by 32-bit bottlenecks like the one of #8460.
Proposal for producing a challenging ISO image:
Produce an ISO image which contains two data files of exactly 2 GiB each:
{{{
xorriso -outdev my_image.iso -add file1 file2 --
}}}
(Note that other than mkisofs -o, this will refuse to overwrite an
existing
image.)
Add a session to the image by adding a small marker file
"this_is_session_2"
{{{
touch this_is_session_2
xorriso -dev my_image.iso -add this_is_session_2 --
}}}
(Note that the second run uses option -dev rather than -outdev. By this it
loads the tree of the first session and adds the new file to it.)
The resulting file "my_image.iso" will bear two sessions. The PVD at
block 16 will direct the reader to a root directory above 4 GiB.
The image will still fit on a single-layer DVD.
--
Comment (by umccullough):
Fix ticket description per comment:1
--
Ticket URL: <http://dev.haiku-os.org/ticket/8473#comment:3>
Haiku <http://dev.haiku-os.org>
Haiku - the operating system.
Other related posts:
