Jerry Amundson wrote:
> (a rare occasion where I top-post... :-)
>
> I'm not familiar with open-iscsi, but how does it determine between
> lun's 0 and 2? Could you be swapping to the filesystem?
I have some udev scripts which present each iscsi target volume and its
LUNs uniquely as, eg:
/dev/iscsi/iqn.2006-12.fileserver:spamassassin/lun0/part
/dev/iscsi/iqn.2006-12.fileserver:spamassassin/lun2/part
These are consistent across all hosts.
> Beyond that, more is needed.. fstab from domU, and it's xen config from
> dom0.
Ok but the fstab is pretty basic :)
> Target recommendations I've seen include:
> InitialR2T No
> ImmediateData Yes
> MaxRecvDataSegmentLength 262144
Ah well I diverge quite a bit in the R2T and ImmediateData settings, in
fact the very reverse of these.
I'd heard that InitialR2T No/ImmediateData Yes was mainly a performance
enhancement? Does it also increase reliability?
Thanks!
> YMMV,
> jerry
>
> On 1/30/07, Steve Wray <steve.wray@xxxxxxxxx> wrote:
>> Hi there,
>>
>> This has just been posted to the following mailing lists:
>> open-iscsi@xxxxxxxxxxxxxxxx
>> iscsitarget-devel@xxxxxxxxxxxxxxxxxxxxx
>>
>> I have been advised to send it to the xen mailing list as well, so here
>> we are!
>> :)
>>
>>
>> I've been testing iscsi for use in a XEN virtualisation environment and
>> have been getting pretty bad filesystem corruption after only about 30
>> minutes of use.
>>
>> I don't have the time or resources to track down exactly what is causing
>> this without some help and guidance; for example its a mystery to me as
>> to whether its the initiator end or the target end or something else
>> again which is causing the problems.
>>
>>
>> 1. The initiator is on a xen3 dom0 host running Debian Etch.
>> 2. The target is provided by another xen3 dom0 host running Debian Etch.
>> 3. The domU (virtual machine) is running Debian Etch and sees the iscsi
>> target as a block device given to it by XEN; the domU knows nothing
>> about iscsi.
>> 4. The filesystem being presented via iscsi is ext3 with default mount
>> options.
>> 5. uname -a on all machines reads pretty much the same:
>> Linux fileserver 2.6.18-3-xen-686 #1 SMP Mon Dec 4 20:48:20 UTC 2006
>> i686 GNU/Linux
>>
>>
>> I get errors like this on the initiator:
>> <errors>
>> Jan 31 09:13:15 xen5 kernel: sd 3:0:0:0: SCSI error: return code =
>> 0x00010000
>> Jan 31 09:13:15 xen5 kernel: end_request: I/O error, dev sde, sector
>> 6996544
>> </errors>
>>
>> and the domU sees its root filesystem disappear and hangs.
>>
>> I see no errors in logs on the machine providing the target.
>>
>> The target daemon was not restarted nor HUPPED during these tests
>> neither was the initiator daemon.
>>
>> The network interfaces on the initiator and target show no errors, no
>> dropped and no overruns (according to ifconfig).
>>
>>
>> My ietd.conf for the iscsi target used by the domU is attached
>> (passwords changed) as is the iscsid node config (again, passwords
>> changed).
>>
>>
>> You will notice that this domU has been running spamassassin; I figured
>> that would give the iscsi layer a good workout since it was running
>> spamassassin for 3 seperate mail servers each with fairly high
>> throughput.
>>
>>
>> I am hoping that there is some config parameter which I have set
>> inappropriately and that there is an easy fix as iscsi + xen would be
>> very useful!
>>
>>
>> If there is any further info that anyone needs please ask.
>>
>> If there are any commandlines I can run to extract info or perform some
>> diagnostics to see where the problems are occuring, please send them to
>> me and I'll send back the results.
>>
>>
>>
>>
>>
>> node.name = iqn.2006-12.fileserver:spamassassin
>> node.transport_name = tcp
>> node.tpgt = 1
>> node.active_conn = 1
>> node.startup = automatic
>> node.session.initial_cmdsn = 0
>> node.session.auth.authmethod = None
>> node.session.auth.username = spamassassin
>> node.session.auth.password = password
>> node.session.timeo.replacement_timeout = 120
>> node.session.err_timeo.abort_timeout = 10
>> node.session.err_timeo.reset_timeout = 30
>> node.session.iscsi.InitialR2T = Yes
>> node.session.iscsi.ImmediateData = No
>> node.session.iscsi.FirstBurstLength = 262144
>> node.session.iscsi.MaxBurstLength = 16776192
>> node.session.iscsi.DefaultTime2Retain = 0
>> node.session.iscsi.DefaultTime2Wait = 0
>> node.session.iscsi.MaxConnections = 1
>> node.session.iscsi.MaxOutstandingR2T = 1
>> node.session.iscsi.ERL = 0
>> node.conn[0].address = 10.10.10.129
>> node.conn[0].port = 3260
>> node.conn[0].startup = automatic
>> node.conn[0].tcp.window_size = 524288
>> node.conn[0].tcp.type_of_service = 0
>> node.conn[0].timeo.logout_timeout = 15
>> node.conn[0].timeo.login_timeout = 15
>> node.conn[0].timeo.auth_timeout = 45
>> node.conn[0].timeo.active_timeout = 5
>> node.conn[0].timeo.idle_timeout = 60
>> node.conn[0].timeo.ping_timeout = 5
>> node.conn[0].timeo.noop_out_interval = 0
>> node.conn[0].timeo.noop_out_timeout = 0
>> node.conn[0].iscsi.MaxRecvDataSegmentLength = 65536
>> node.conn[0].iscsi.HeaderDigest = CRC32C
>> node.conn[0].iscsi.DataDigest = CRC32C
>> node.conn[0].iscsi.IFMarker = No
>> node.conn[0].iscsi.OFMarker = No
>>
>>
>>
>> Target iqn.2006-12.fileserver:spamassassin
>> Lun 0 Path=/dev/volumes/spamassassin,Type=fileio
>> Lun 2 Path=/dev/volumes/spamassassin-swap,Type=fileio
>>
>> Alias spamassassin
>>
>> IncomingUser spamassassin password
>>
>> InitialR2T Yes
>> ImmediateData No
>> MaxRecvDataSegmentLength 8192
>> MaxXmitDataSegmentLength 8192
>> MaxBurstLength 262144
>> FirstBurstLength 65536
>> DefaultTime2Wait 2
>> DefaultTime2Retain 20
>> MaxOutstandingR2T 8
>> DataPDUInOrder Yes
>> DataSequenceInOrder Yes
>> ErrorRecoveryLevel 0
>> HeaderDigest CRC32C
>> DataDigest CRC32C
>>
>>
>>
>>
>> _______________________________________________
>> Xen-users mailing list
>> Xen-users@xxxxxxxxxxxxxxxxxxx
>> http://lists.xensource.com/xen-users
>>
>>
>
>
kernel = "/boot/vmlinuz-2.6.18-3-xen-686"
ramdisk = "/boot/initrd.img-2.6.18-3-xen-686"
memory = 256
name = "spamassassin-iscsi"
vcpus = 1
vif = [ 'mac=AA:00:00:07:47:AD, bridge=xenbr0' ]
disk = [ 'phy:iscsi/iqn.2006-12.fileserver:spamassassin/lun0/part,sda1,w',
'phy:iscsi/iqn.2006-12.fileserver:spamassassin/lun2/part,sda2,w']
root = "/dev/sda1 ro"
proc /proc proc defaults 0 0
/dev/sda1 / ext3 defaults 0 1
/dev/sda2 none swap sw 0 0
_______________________________________________
Xen-users mailing list
Xen-users@xxxxxxxxxxxxxxxxxxx
http://lists.xensource.com/xen-users
|
Home