pg.ddx.io pgsql-performance@postgresql.org mailing list archive
help / color / mirror / Atom feedAutodetect of software RAID1+0 fails
6+ messages / 5 participants
[nested] [flat]
* Autodetect of software RAID1+0 fails
@ 2007-06-01 17:57 Craig James <craig_james@emolecules.com>
0 siblings, 2 replies; 6+ messages in thread
From: Craig James @ 2007-06-01 17:57 UTC (permalink / raw)
To: ; +Cc: pgsql-performance
Apologies for a somewhat off-topic question, but...
The Linux kernel doesn't properly detect my software RAID1+0 when I boot up. It detects the two RAID1 arrays, the partitions of which are marked properly. But it can't find the RAID0 on top of that, because there's no corresponding device to auto-detect. The result is that it creates /dev/md0 and /dev/md1 and assembles the RAID1 devices on bootup, but /dev/md2 isn't created, so the RAID0 can't be assembled at boot time.
Here's what it looks like:
$ cat /proc/mdstat
Personalities : [raid0] [raid1]
md2 : active raid0 md0[0] md1[1]
234436224 blocks 64k chunks
md1 : active raid1 sde1[1] sdc1[2]
117218176 blocks [2/2] [UU]
md0 : active raid1 sdd1[1] sdb1[0]
117218176 blocks [2/2] [UU]
$ uname -r
2.6.12-1.1381_FC3
After a reboot, I always have to do this:
mknod /dev/md2 b 9 2
mdadm --assemble /dev/md2 /dev/md0 /dev/md1
mount /dev/md2
What am I missing here?
Thanks,
Craig
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autodetect of software RAID1+0 fails
@ 2007-06-01 18:51 Dimitri <dimitrik.fr@gmail.com>
parent: Craig James <craig_james@emolecules.com>
1 sibling, 1 reply; 6+ messages in thread
From: Dimitri @ 2007-06-01 18:51 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
Craig,
to make things working properly here you need to create a config file
keeping both raid1 and raid0 information (/etc/mdadm/mdadm.conf).
However if your root filesystem is corrupted, or you loose this file,
or move disks somewhere else - you are back to the same initial issue
:))
So, the solution I've found 100% working in any case is: use mdadm to
create raid1 devices (as you do already) and then use LVM to create
raid0 volume on it - LVM writes its own labels on every MD devices and
will find its volumes peaces automatically! Tested for crash several
times and was surprised by its robustness :))
Rgds,
-Dimitri
On 6/1/07, Craig James <craig_james@emolecules.com> wrote:
> Apologies for a somewhat off-topic question, but...
>
> The Linux kernel doesn't properly detect my software RAID1+0 when I boot up.
> It detects the two RAID1 arrays, the partitions of which are marked
> properly. But it can't find the RAID0 on top of that, because there's no
> corresponding device to auto-detect. The result is that it creates /dev/md0
> and /dev/md1 and assembles the RAID1 devices on bootup, but /dev/md2 isn't
> created, so the RAID0 can't be assembled at boot time.
>
> Here's what it looks like:
>
> $ cat /proc/mdstat
> Personalities : [raid0] [raid1]
> md2 : active raid0 md0[0] md1[1]
> 234436224 blocks 64k chunks
>
> md1 : active raid1 sde1[1] sdc1[2]
> 117218176 blocks [2/2] [UU]
>
> md0 : active raid1 sdd1[1] sdb1[0]
> 117218176 blocks [2/2] [UU]
>
> $ uname -r
> 2.6.12-1.1381_FC3
>
> After a reboot, I always have to do this:
>
> mknod /dev/md2 b 9 2
> mdadm --assemble /dev/md2 /dev/md0 /dev/md1
> mount /dev/md2
>
> What am I missing here?
>
> Thanks,
> Craig
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autodetect of software RAID1+0 fails
@ 2007-06-01 19:54 Luke Lonergan <llonergan@greenplum.com>
parent: Dimitri <dimitrik.fr@gmail.com>
0 siblings, 0 replies; 6+ messages in thread
From: Luke Lonergan @ 2007-06-01 19:54 UTC (permalink / raw)
To: Dimitri <dimitrik.fr@gmail.com>; Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
Dimitri,
LVM is great, one thing to watch out for: it is very slow compared to pure
md. That will only matter in practice if you want to exceed 1GB/s of
sequential I/O bandwidth.
- Luke
On 6/1/07 11:51 AM, "Dimitri" <dimitrik.fr@gmail.com> wrote:
> Craig,
>
> to make things working properly here you need to create a config file
> keeping both raid1 and raid0 information (/etc/mdadm/mdadm.conf).
> However if your root filesystem is corrupted, or you loose this file,
> or move disks somewhere else - you are back to the same initial issue
> :))
>
> So, the solution I've found 100% working in any case is: use mdadm to
> create raid1 devices (as you do already) and then use LVM to create
> raid0 volume on it - LVM writes its own labels on every MD devices and
> will find its volumes peaces automatically! Tested for crash several
> times and was surprised by its robustness :))
>
> Rgds,
> -Dimitri
>
> On 6/1/07, Craig James <craig_james@emolecules.com> wrote:
>> Apologies for a somewhat off-topic question, but...
>>
>> The Linux kernel doesn't properly detect my software RAID1+0 when I boot up.
>> It detects the two RAID1 arrays, the partitions of which are marked
>> properly. But it can't find the RAID0 on top of that, because there's no
>> corresponding device to auto-detect. The result is that it creates /dev/md0
>> and /dev/md1 and assembles the RAID1 devices on bootup, but /dev/md2 isn't
>> created, so the RAID0 can't be assembled at boot time.
>>
>> Here's what it looks like:
>>
>> $ cat /proc/mdstat
>> Personalities : [raid0] [raid1]
>> md2 : active raid0 md0[0] md1[1]
>> 234436224 blocks 64k chunks
>>
>> md1 : active raid1 sde1[1] sdc1[2]
>> 117218176 blocks [2/2] [UU]
>>
>> md0 : active raid1 sdd1[1] sdb1[0]
>> 117218176 blocks [2/2] [UU]
>>
>> $ uname -r
>> 2.6.12-1.1381_FC3
>>
>> After a reboot, I always have to do this:
>>
>> mknod /dev/md2 b 9 2
>> mdadm --assemble /dev/md2 /dev/md0 /dev/md1
>> mount /dev/md2
>>
>> What am I missing here?
>>
>> Thanks,
>> Craig
>>
>> ---------------------------(end of broadcast)---------------------------
>> TIP 1: if posting/reading through Usenet, please send an appropriate
>> subscribe-nomail command to majordomo@postgresql.org so that your
>> message can get through to the mailing list cleanly
>>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 3: Have you checked our extensive FAQ?
>
> http://www.postgresql.org/docs/faq
>
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autodetect of software RAID1+0 fails
@ 2007-06-01 21:35 Steinar H. Gunderson <sgunderson@bigfoot.com>
parent: Craig James <craig_james@emolecules.com>
1 sibling, 2 replies; 6+ messages in thread
From: Steinar H. Gunderson @ 2007-06-01 21:35 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
On Fri, Jun 01, 2007 at 10:57:56AM -0700, Craig James wrote:
> The Linux kernel doesn't properly detect my software RAID1+0 when I boot
> up. It detects the two RAID1 arrays, the partitions of which are marked
> properly. But it can't find the RAID0 on top of that, because there's no
> corresponding device to auto-detect. The result is that it creates
> /dev/md0 and /dev/md1 and assembles the RAID1 devices on bootup, but
> /dev/md2 isn't created, so the RAID0 can't be assembled at boot time.
Either do your md discovery in userspace via mdadm (your distribution can
probably help you with this), or simply use the raid10 module instead of
building raid1+0 yourself.
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autodetect of software RAID1+0 fails
@ 2007-06-02 00:23 mark@mark.mielke.cc
parent: Steinar H. Gunderson <sgunderson@bigfoot.com>
1 sibling, 0 replies; 6+ messages in thread
From: mark@mark.mielke.cc @ 2007-06-02 00:23 UTC (permalink / raw)
To: Craig James <craig_james@emolecules.com>; pgsql-performance
> On Fri, Jun 01, 2007 at 10:57:56AM -0700, Craig James wrote:
> > The Linux kernel doesn't properly detect my software RAID1+0 when I boot
> > up. It detects the two RAID1 arrays, the partitions of which are marked
> > properly. But it can't find the RAID0 on top of that, because there's no
> > corresponding device to auto-detect. The result is that it creates
> > /dev/md0 and /dev/md1 and assembles the RAID1 devices on bootup, but
> > /dev/md2 isn't created, so the RAID0 can't be assembled at boot time.
Hi Craig:
I had the same problem for a short time. There *is* a device to base the
RAID0 off, however, it needs to be recursively detected. mdadm will do this
for you, however, if the device order isn't optimal, it may need some help
via /etc/mdadm.conf. For a while, I used something like:
DEVICE partitions
...
ARRAY /dev/md3 level=raid0 num-devices=2 UUID=10d58416:5cd52161:7703b48e:cd93a0e0
ARRAY /dev/md5 level=raid1 num-devices=2 UUID=1515ac26:033ebf60:fa5930c5:1e1f0f12
ARRAY /dev/md6 level=raid1 num-devices=2 UUID=72ddd3b6:b063445c:d7718865:bb79aad7
My symptoms were that it worked where started from user space, but failed during
reboot without the above hints. I believe if I had defined md5 and md6 before
md3, it may have worked automatically without hints.
On Fri, Jun 01, 2007 at 11:35:01PM +0200, Steinar H. Gunderson wrote:
> Either do your md discovery in userspace via mdadm (your distribution can
> probably help you with this), or simply use the raid10 module instead of
> building raid1+0 yourself.
I agree with using the mdadm RAID10 support. RAID1+0 has the
flexibility of allowing you to fine-control the RAID1 vs RAID0 if you
want to add disks later. RAID10 from mdadm has the flexibility that
you don't need an even number of disks. As I don't intend to add disks
to my array - the RAID10 as a single layer, with potentially better
intelligence in terms of performance, appeals to me.
They both worked for me - but I am sticking with the single layer now.
Cheers,
mark
--
mark@mielke.cc / markm@ncf.ca / markm@nortel.com __________________________
. . _ ._ . . .__ . . ._. .__ . . . .__ | Neighbourhood Coder
|\/| |_| |_| |/ |_ |\/| | |_ | |/ |_ |
| | | | | \ | \ |__ . | | .|. |__ |__ | \ |__ | Ottawa, Ontario, Canada
One ring to rule them all, one ring to find them, one ring to bring them all
and in the darkness bind them...
http://mark.mielke.cc/
^ permalink raw reply [nested|flat] 6+ messages in thread
* Re: Autodetect of software RAID1+0 fails
@ 2007-06-02 05:41 Luke Lonergan <llonergan@greenplum.com>
parent: Steinar H. Gunderson <sgunderson@bigfoot.com>
1 sibling, 0 replies; 6+ messages in thread
From: Luke Lonergan @ 2007-06-02 05:41 UTC (permalink / raw)
To: Steinar H. Gunderson <sgunderson@bigfoot.com>; Craig James <craig_james@emolecules.com>; +Cc: pgsql-performance
Steinar,
On 6/1/07 2:35 PM, "Steinar H. Gunderson" <sgunderson@bigfoot.com> wrote:
> Either do your md discovery in userspace via mdadm (your distribution can
> probably help you with this), or simply use the raid10 module instead of
> building raid1+0 yourself.
I found md raid10 to be *very* slow compared to raid1+0 on Linux 2.6.9 ->
2.6.18. Very slow in this case is < 400 MB/s compared to 1,800 MB/s.
- Luke
^ permalink raw reply [nested|flat] 6+ messages in thread
end of thread, other threads:[~2007-06-02 05:41 UTC | newest]
Thread overview: 6+ messages (download: mbox mbox.gz follow: Atom feed)
-- links below jump to the message on this page --
2007-06-01 17:57 Autodetect of software RAID1+0 fails Craig James <craig_james@emolecules.com>
2007-06-01 18:51 ` Dimitri <dimitrik.fr@gmail.com>
2007-06-01 19:54 ` Luke Lonergan <llonergan@greenplum.com>
2007-06-01 21:35 ` Steinar H. Gunderson <sgunderson@bigfoot.com>
2007-06-02 00:23 ` mark@mark.mielke.cc
2007-06-02 05:41 ` Luke Lonergan <llonergan@greenplum.com>
This inbox is served by DDX for PostgreSQL; see mirroring instructions
for how to clone and mirror all data and code used for this inbox