bunderthey do? why the long string to turn features on and off in the gentoo docs?
DeHackEdfeatures can only be turned on, not off depending on the features. some can be effectively turned off by destroying all datasets affects. some are stuck on once enabled
bunderyou know what i mean :P
DeHackEd-d turns all features off at creation time. then you can 'zpool set feature@which_one=enabled poolname'
DeHackEdbut 'set enabled' is the only format allowed
bunder-o feature@async_destroy=enabled -o feature@empty_bpobj=enabled -o feature@lz4_compress=enabled -o feature@spacemap_histogram=enabled -o feature@enabled_txg=enabled -o feature@extensible_dataset=enabled -o feature@bookmarks=enabled
bunderall the unspecified ones are off
DeHackEdsame as doing 'zpool set...' at creation time
DeHackEdso, yeah...
bunderbut you just said they're all on by default now
DeHackEdby default
DeHackEdbut your creation command has -d in it, right?
bunderoh it did
bundersorry, i'm like half awake
bundernight shift didn't show up and they hauled me out of bed
gotwfnight shit happens....
DeHackEdtell me about it...
Shinigami-Samaoh man I just read the hole birth git thread... scary stuff
Shinigami-SamaI'm glad a handle got put on it so quickly though
PMTdjs: "mostly" have one - still need to refine it and add a couple of things, like avoiding false positives on pools that had hole_birth turned on after initial creation
PMToh boy, i think i get to parse zhack output
faenilI read so many threads/posts about LUKS under zfs...is there anyone here who has a "trusted" one? or who actually knows the internals and can clear my doubts? :D
bunderabout time freenode
Boobuigifaenil: I run ZFS on top of LUKS and wouldn't have it any other way.
faenilBoobuigi: not concerned by the "redundancy above encryption" ?
PMTfaenil: you could also wait and hope that https://github.com/zfsonlinux/zfs/pull/4329 lands soon
faenilPMT: yeah, I've seen that, I think I'll keep an eye on that and check it out again in a few months :)
faenilPMT: meanwhile I'm considering LUKS in a VM inside zfs
PMTwhy in a VM?
faenilfor convenience
bundercannot receive incremental stream: destination blah has been modified since most recent snapshot
bunderand i deleted that snapshot too
bundercan i copy a snapshot to another snapshot
PMTbunder: you could presumably do something like zfs rollback blah@most_recent
PMTor what, in particular, do you want to do?
bunderyeah i rolled back the destination but i don't have the source to send again
faenilPMT: got any advice/guide on how to handle VMs with zfs? I read the recommendation is to have one dataset per vm so you can roll them back separately
faenil(and back them up separately?)
fasdfvirtual machines?
PMTfaenil: the two basic ways of doing it that i'm aware of would be a zvol per VM or a dataset per VM. I'd go with the latter.
faenilwhy? (out of curiosity)
PMTexcuse me, i misused dataset to mean filesystem there
fasdfyou're talking about linux vms, right?
PMTbut mostly because otherwise snapshots get expensive
faenilfasdf: yes, does it make any diff?
faenilPMT: mm I see, what should I read up to get a better view over that?
fasdfI dunno. I was thinking recently how can I adapt zfs to be useful instead of qcow2 disk images I have laying somewhere
fasdfbut mine vms are windows and osx
fasdfmaybe lvm is a better approach in my case
PMTfaenil: when i say "snapshots get expensive" I mean that having snapshots of a zvol means you have a reservation of sizeof(zvol) atop any existing storage occupied by the zvol, because most systems you'd hand a zvol to don't really have any idea what "out of space" would mean when they try to rewrite a block
PMTfasdf: i mean, qcow2 or w/e disk images on a zfs fs is fine
fasdfwhat's w/e
faenilPMT: and how is it different when snapshotting a FS?
PMTw/e: whatever
PMTfaenil: it doesn't require such a reservation, so you could indeed get back "out of space" when trying to rewrite a block on a VM image. ;)
fasdfPMT: what's your opinion on [ qcow2 images on zfs ] vs [ lvm solution ] here
PMTfasdf: what kind of LVM solution would you propose? AFAIK the only things LVM would offer you would either be a block device (which zfs could do as a zvol) or a filesystem on top of an LV, which could have any of the same behaviors as images on any other filesystem (like ZFS). What're you trying to optimize for?
faenilPMT: and how bad it is if most of the space on the disk is occupied by the VM .vdi?
faenilhow bas is it*
faenilthat shouldn't make a different iiuc because zfs operates on block-level, iiuc
faenildifference*, argh
PMTfaenil: I mean, like any other filesystem, above some threshold of usage, performance will be reduced because you're scrambling around to try and find free space to write things, which also means you need to scramble around to read it back later. ZFS has various methods it uses to minimize the impact of this, but if there's only so much space free, you can only do so much to mitigate it. or did you mean so
PMTme other kind of implications?
faenilother, yeah
bunderbah, forget it... i renamed monday to sunday and sent that instead, i can deal with a missing daily this week heh
faenilPMT: I'm more thinking if it's actually worth having a VM (it kind of gives me the idea that everyone is properly separated and I can move it around or back it up as needed)
fasdfPMT: I didn't know much about zvol, sorry
faenilgah, wtf is wrong with my english today
PMTfaenil: like what? I mean, if you rewrite the entire contents of the vdi regularly (or a good portion of it), and use snapshots, if the pool doesn't have space for it, you'll run out quickly.
PMTfasdf: it's fine, i'm just trying to understand what benefit/drawback you're trying to get out of LVM or ZFS or ...
fasdfPMT: so I can just attach a zvol to kvm instance and create guid parition table and hfs+ volumes?
faenilPMT: doesn't that also apply to the same situation without VM? if you rewrite most of the files, you'll have the same problem, right?
faenilbut yes, I'm looking for those kind of implications
PMTfaenil: of course. but with a VM, the OS running in the VM doesn't exactly have a way to be told "I'm out of space" effectively, particularly because the helpless VM has no way of freeing space from inside itself.
faenilmm right
PMTfasdf: yeah, just like if you handed $VM_SOFTWARE a file on disk or a raw block device that goes to an actual hard drive or ...
fasdfsounds hot
PMTI'd probably use files on a filesystem on ZFS instead of zvols unless I had a particular reason, just like I would for LVM, because I can more easily shuffle files around different platforms than I can block devices, without turning them into files.
PMTI'm far from a grizzled old veteran of this, though, so you should definitely think about whether you have any compelling reason to use one or the other.
faenilPMT: on a completely different note, what is the best practice for backing up to an untrusted server?
PMTfaenil: in what context? I mean, it depends on what you're trying to be prepared for, and how far you're willing to go out of your way. for example, you could use a GPG private key (or a yubikey or some other hardware key device) to encrypt+sign your shit and shuffle it off to $UNTRUSTED_SERVER, and then decrypt+verify it on restoring, but that's somewhat cumbersome in a few ways
faenilwhat do you mean by "shuffle it off" ?
PMTtransmit the encrypted+signed bits to $UNTRUSTED_SERVER in whatever means you were planning to use - scp, rsync, $ARBITRARY_PROTOCOL
dasjoegpg | nc
PMTfor example.
faenilyeah sure
faenildasjoe: yeah I read zfs send | gpg is one of the most widespread
PMTfaenil: of course, the trick to that is that you need to then be sure to keep your key both secure and resiliently stored, so you don't end up having a storage or hw key device failure and locked out of all your backups forever. ;)
dasjoeI probably would not do that, though. I'd rather try get the remote host to export a (possibly simulated) block device so I can throw luks and a zpool on it
faenilPMT: of course. But doesn't it always boil down to that?
PMTdasjoe: then what, send|recv $LUKS_ZPOOL
PMTfaenil: yes, but just because it's always true doesn't mean the person asking knows it.
faenilsure :)
dasjoePMT: yeah, I like having the full snapshot history and easy incremental upgrades
PMTno argument here, just trying to understand the workflow you're proposing.
faenildasjoe: yeah I'd like to go with send | recv if possible
PMThow much luck have you had convincing backup providers to expose rbd/iscsi/... block devices over the WAN though :)
dasjoeMaybe it's possible to run aoe in userspace and wrap it with some encryption, so you can somewhat safely access the block device
PMTi'm crying tears of blood about ATAoE right now
dasjoePMT: oh, commercial backup providers. Not a fan, but mostly because I'm thinking about ways how to (and trying to) get into that business
dasjoeSo, envy
PMTI mean, the space is pretty saturated with various options. I'd be curious to see what niche you managed to carve.
dasjoehttps://airtower.wordpress.com/2010/06/08/nbd-through-ssh/ ← heh
dasjoePMT: I'll let you know once we're past the early prototype stages. Right now the idea exists as workflow charts and some preliminary hardware on my desk, next to some client's horribly broken server
bunderthat nbd sounds an awful lot like iscsi
dasjoebunder: looks nicer, though. Fully userspace server, can listen on localhost and then tunnel through ssh
bunderdunno, i could go either way (unless linux doesn't to it natively)
PMTdasjoe: you could always speak 9p ;)
faenildasjoe: although send|recv means sending unencrypted data right?
faenilwell, sending an unencrypted stream of encrypted data
dasjoefaenil: that's why you export the remote block device via iscsi/aoe/nbd and connect it to your local machine. Your *local* machine encrypts it with luks, then puts a pool in it
dasjoeSo, your local machine does both the send and recv, all the remote server sees is fully encrypted writes to its exported block device
faenilmmm, interesting
faenilalthough I know nothing about iscsi/aoe/nbd :D but that sounds like a good workflow
faenilthat works fine for backing up to a "trusted" partner
faenilwhich is still infinitely better than not backing up :)
dasjoeOh look, a userspace nbd server with tls support
dasjoefaenil: why just to a trusted partner? You don't care about the remote end reading your disk or anything, as you run luks yourself
faenildasjoe: sorry I did not mean trusted...I meant something you could have access to, in order to setup the block device export, which otherwise you will probably not get from commercial backup/cloud services, as PMT was saying
dasjoeYes, right
dasjoePMT: well, it may be time to build and provide such a service, acutally
dasjoeWhat about giving people access to zvol-backed "devices" remotely, you can easily change its size, too
faeniland pay based on the space you use :)
faenil(although that's dynamic, so it's tricky)
faenildasjoe: but if you have a VM with LUKS you can just rsync the .vdi to an untrusted server, can't you
dasjoeYes, but that way you'd have to have local space for that VM
faenilwhat do you mean?
faenilyou need space for the VM in any case, don't you?
PMTdasjoe: i mean, why back with zvol versus files?
faenilbdsync more than rsync
dasjoePMT: why wouldn't you use zvols?
PMTnot wanting to pay the snapshot reservation if you didn't have to
dasjoeUse sparse zvols, if you want. Or just don't create snapshots. Or have clients pay for snapshots as if they were regular data ;)
faenildasjoe: what did you mean by "you have to have local space for that VM"?
dasjoefaenil: I assumed you wanted to zfs recv in a vm, so you would need space for that
faenilbut you need space for anything, don't you
dasjoeNot with an untrusted remote host that's giving you block devices
faenilwhy not?
faenilif the VM disk does not have a fixed size, then there's no wasted space
dasjoeBecause you'd connect to the remote block device, then put your backups on it as if it was local. No need for local backup space, then
faenildasjoe: the same way you'd just sync your vdi to an untrusted server, I don't see why that requires a local copy of the vm
dasjoefaenil: yes, we were talking about different things. I assumed you wanted to use a VM with luks as a backup target for some datasets, not backup the VM
faenilah ok
[gnubie]ACTION waves
[gnubie]anyone can give me a hint how to fix this problem => https://imagebin.ca/v/2nQ8DiqwtsQ8
ptx0swap on ZFS?
ptx0you're asking for troubl
DeHackEd[gnubie]: no logs from systemd? it says successful import but failed mount.
DeHackEdusually the reason mounts would fail is the mount point is not empty
DeHackEdcan also examine for reference: zfs list -o name,mounted,mountpoint
[gnubie]that’s zfs on / filesystem.
[gnubie]i followed https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-ZFS
[gnubie]i cannot see the logs or proceed on executing any command because i’m stuck on that screenshot
Nukienfaenil, Check out Jim Salter's blog - lots of good info in there - http://jrs-s.net/2016/06/29/verifying-copies/. Also Aaron Toponce - https://pthree.org/
faenilNukien: you're bringing bad news eh? :P ok, thanks for the link ;)
NukienIs zfs smart enough to mount stuff in the correct order ? For example, having 2 datasets, one mounted on a subdir in the other.
Nukiendataset A mounted on /stuff, dataset B mounted on /stuff/other
NukienWill it mount them in the correct order, A first, then B ? Or is it luck of the draw ?
NukienI've tested a bit, and so far it seems good, but would like assurance, otherwise I have to tweak things to make sure they mount in the correct order.
Lalufuif you let zfs do the mounting then yes
Lalufuif you do your mounting yourself then it's your problem, of course
Nukien'cuz it would suck to have B mount first, creating /stuff/other, then A fail to mount since /stuff isn't empty
NukienLalufu, Not doing anything specific yet, just letting the stock ubuntu 16.04 and Cent7 install/configs handle it.
PMTNukien: i haven't tested if you try it with something like dataset foo/bar and foo/baz, with foo/baz being mounted at /foo/bar/baz, but otherwise yes
faenilNukien: your script does not setup an encrypted boot, does it?
Nukienfaenil, No encrypted /boot, no.
NukienYou can put /boot on a usb stick though
NukienI was heading that way with detached luks headers, but don't have a clean way to use the detached headers (yet)
faenilyeah, although not very feasible for a server
NukienAt least with stock initramfs scripts etc
NukienSpin up a VM with 3x disks, about 1G each. Test the script in the various scenarios, see which works best for you
Nukienmay 2g each
Nukienmaybe 2g each that is
NukienI like the dropbear functionality - ssh in to input the passphrase before full boot
NukienAnyway, bang on it. Find problems. Lemme know.
faenilNukien: honestly, I think I'm going to try a simpler way first, the script does too much, and I don't like executing script without knowing what they do, and that would take too long :)
faenilso I'll have to settle with a subset of the feature, but knowing what I'm typing :)
NukienHah. Well, hence my saying play with it in a VM.
faenilNukien: not saying it won't work, just that the time it takes to understand what it's doing is more than what I have available at the moment...I could pick it up in the future, when I have more time and a better knowledge of luks/zfs/etc...all is new now, it's overwhelming
faenilNukien: but my first feedback is, it will be more appealing if it's easier to pick up, i.e. all the cool advanced stuff is hidden behind a SPOILER :)
faeniland you did a great job, btw, not trying to deny that in any way :)
NukienHeh no need for flattery. What do you mean hidden behind a spoiler ?
NukienMore comments ?
faenilNukien: no seriously, the script is great...I just don't have enough time to know what it's doing, so that I can fix my system if something breaks. Scripts are cool to set things up, but if something breaks then you have to know how to put the pieces back together :)
faenilNukien: I mean like having a wiki where all the additional scripts are hidden in an Advanced section, for instance
faenilto separate the core of the job for the "utilities"
faenildropbear, the mbuffer stuff, the faileddisk etc are all really useful scripts, but maybe not everyone needs them
faenilit's probably just because all that stuff is in the same file on pastebin, that's all I'm saying :) I don't know how many of those pieces work, and have to research every bit
faenilI don't know how a lot of those pieces*
BaughnI have a ZFS pool (2x SSD, mirrored) which has degraded over time, and just a few hours ago outright *hung* during heavy load.
BaughnAs in, no i/o completes at all.
BaughnThis does make it a little difficult to log in. But if I can reproduce it, is there any mechanism I can use to see whether the cause is the SSDs or ZFS?
DeHackEd`iostat -dmx 1` and `dmesg`
BaughnNow let's just hope a reboot fixes it.
BaughnOn another note, scrubbing a 10T array at 8MB/s is Very. Slow.
BaughnIs there any way to speed this up? :X
DeHackEdfigure out why it's so slow and fix it.
GrayShadeBaughn: there are some tunables which might help a bit
GrayShadebut probably won't get you from 8 to 100 MB/s
BaughnIt's sped up to 100 MB/s already, which is probably the best I can hope for.
GrayShadeBaughn: https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSScrubsOurSpeedup
BaughnSo now I'm more curious why it starts so slow.
BaughnACTION will still read the article, of course~
GrayShadei think scrubs follow the directory tree, so the speed can depend on whethere you have lots of small files or few larger ones
BaughnI kinda wish I had dtrace, now.
faenilNukien: is https://unixsysdoc.wordpress.com your blog?
NukienNope - not me. Good blog though
dasjoeNukien: you're using mdadm --metadata=1.0, why?
dasjoeNukien: also, it's --raid-devices=n, not --raid-device=n ;)
dasjoeNukien: you copied that part from fearedbliss's Gentoo notes, right? Should not apply here, we're not using extlinux
faenildasjoe: the grub installation here fails with "failed to get canonical path for" https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-ZFS
faenilI've tried it twice, without LUKS, and now with LUKS below zfs
faenildo you know why?
dasjoeNot yet, you didn't pastebin any log ;)
faenildasjoe: syslog? grub install log?
sveinseptx0: Swap on ZFS = trouble? What is lesser of these evils: swap on zfs/raidz2 or swap on a slooow USB stick or no swap at all (16GB ram on a 6x3TB raidz2 pool)
BaughnNo swap at all.
BaughnIIRC swap-on-zvol works better than it used to, but don't take my word for that. Also, swap in general is almost pointless unless you put it on ssd.
ptx0only pointless if ou don't mind processes being killed by OOMkiller
sveinseCurrently I have 8 GB RAM on this server, and I've setup swap on zfs. I see that it swaps out 200-300Mb consistently. Thus new memory, 16GB, it's on its way....
sveinseIt's amazing how much memory a java application wants... (creashplan in this case)
faenildasjoe: I don't remember how I got it to work, eventually...yesterday :/
ptx0even with 96G RAM I still use swap
dasjoefaenil: ok, I'll try to help. For a start, lte's look at your partitions: sgdisk -p /dev/sdX |& curl -F 'sprunge=<-' http://sprunge.us
faenildasjoe: if there's something I haven't read, please redirect me to it, I don't want to bother ;)
dasjoefaenil: it's ok, I'm writing up some stuff for my own installations right now, so I'm somewhat absent
faenildasjoe: I have 2 disks, 2 partitions each, one unformatted, one ext2 for boot. I created a mirror from the 2 part1
faenilsgdisk shows 2 parts with code 8300
faenil(for each disk)
faenilno matter what path I give to grub-install, it always returns "failed to get canonical path for /dev/sdb-enc"
faenilfirst: I have /dev/mapper/sdb-enc and /dev/mapped/sdc-enc, not sure why grub is looking at /dev/sdb-enc
faenilsecond: why is it looking at a path which is not the one I'm installing to :/
faenilif I try grub-install /dev/sdb, same error, it looks at /dev/sdb-enc
dasjoeOh right, you'd need to symlink /dev/mapper/sdb-enc and sdc-enc to /dev/
faenilmmm...how comes?
dasjoeBug in grub, it's looking for its devices under /dev alone
faenilbut it shouldn't install in sdb-enc anyway, that's the luks partition
faenildasjoe: do you have a link to the bug? or know where to find it? the threads on zfsonlinux are huge :)
gotwffaenil: see http://www.teamcool.net for enlightenment regarding canonical path error
faenilI found a couple of related grub bugs but they seem to be marked as fixed in 2015
faenilyeah I've seen that
faenilgotwf: cheers
gotwffaenil: basically boils down to you need to fool grub by creating your own symlinks to what it is expecting to find.
dasjoefaenil: https://bugs.launchpad.net/ubuntu/+source/zfs-linux/+bug/1527727
faenilgotwf: yeah I just wanted the link to the upstream bg so that I could write it down in my notes and track it :)
faenildasjoe: thanks
gotwffaenil: in this regard, there's more that one way to do it...
faenilso is that why tutorials are full of udev rules?
faenilwell not full...
gotwfPartly. One way to do it.
gotwfI just create them manually as needed as a double check against accidental updates, i.e. asleep at the switch
gotwfcuz if they don't exist, the update will fail and hit me with a clue bat
gotwfLess than elegant? But keeps me in control.
gotwfMoreover, seeing how udev is systemd inbred shite, you cannot trust it.
faenilmmm, ubuntu's shipped zfs comes with a custom udev rule...although I'm installing xenial :/
faenilanother one hit https://savannah.gnu.org/bugs/?41524
faenilnow "this gpt partition contains no BIOS boot partition
sveinsefaenil: Yep, you have GPT partitioning I suppose
sveinseThen you can't just install grub in the MBR any more, you need a dedicated partition for it
faeniloh, I see. that's the bit I was missing then, I did not get why I had a custom boot partition. It all makes sense now
faenilext2 does not support embedding...
faenilargh, alright
sveinseI formatted my pool with "whole disks" back in the days, which implies GPT and a fixed partition setup. Consequently, I have an USB-device for GRUB and /boot storage, while / is on zfs
faenilmakes sense
faeniland what did you use for grub? fat32?
sveinseThe first partition starts at 2048, so it should technically be possible to add grub_boot partition, and with grub supporting zfs, get /boot from zfs. But I haven't risked changing the working pool (being production and all)
gotwfACTION fwiw, I don't like embedding in MBR gap. More brittle/skeetchy. If your hw can BIOS boot from an gpt formatted disk, just make a small ef02 partition for grub.
sveinseMy /boot is ext4
sveinsefaenil: The USB drive use DOS partitioning, so the first stage grub loader sits in the traditional MBR. /boot is ext4 as I just said
sveinseIf you use the grub_boot (ef02) partition, I don't think you format it afaik, it's a raw partition. Perhaps I'm wrong
djs scan: scrub repaired 32K in 7h24m with 0 errors on Sat Jul 9 20:53:58 2016
faenilnot sure where I got the advice to use ext2 for boot :/
djsbut no CKSUM errors listed... anyone know how I can get details on what was repaired?
faenilah, I think it came from https://wiki.archlinux.org/index.php/Dm-crypt/Encrypting_an_entire_system#Preparing_the_boot_partition_2
faenil"embedding is not possible but this is required for RAID or LVM"
djsfaenil: I've been wondering about converting my existing zfs pools to zfs on lucks as well, but I have /boot on zfs in the pools so it seems like even more fun
djsnot impossible afaict
faenilit's amazing there's no clean reproducible guide to have Ubuntu on LUKS
djsheh, the default ubuntu install allows you to encrypt which ends up as lvm on luks
faenilit's full of tutorials, none of them current or complete
faenilyeah, no zfs
djsyeah, there's that
djsfaenil: http://www.pavelkogan.com/2014/05/23/luks-full-disk-encryption/ makes it sound like encrypted boot should be ok
djspresumably if /boot was in zfs on luks that should still be ok
faenilthat would be ideal, I wanted to get something working first
faenilthat's using msdos partition tablet
djsyeah that's no good... need GPT
faenilI formatted to ext4, and still says ext2 does not support embedding .|
faenilgotwf: ^ ideas?
gotwfHm... this was a do over from scratch? Or a mbr to gpt conversion?
bundercheck your kernel, i think the ext4 driver also does ext2/3, i know if i have all 3 turned on my one ext system, it mounts ext3 as ext2 for no seemingly good reason
gotwfSo, you have a small, e.g. 1MiB partition for BIOS grub, type ef02? Plus then another, e.g. 256MiB, ext2 for /boot?
faenilbut it's not mounted
faenilgotwf: ahm, nope, just 1 huge luks block followed by ext4
gotwfI don't know much about luks, as I have never used it. No need here and hence avoided the additional complexity.
faenilgotwf: boot is currently separated from luks, so your knowledge should still apply
faenilI guess I'm missing the additional 1MiB partition...
gotwfIf someone breaks into my home and gets physical access, I have bigger problems. If I happen to be home a the time, they're about to have bigger problems.
faenilI do this as a learning excercise, most importantly
gotwfMy kid busted my lappie. I don't really want/need mobile.
gotwfI spend enough time behind computer as is. If I am out and about, I need to enforce that break.
faenilI totally understand you :P (software developer here)
gotwffaenil: Rodsbooks is one of the best sites I have found for gpt, uefi, boot loader stuff
faenilok :)
gotwffrom memory, http://rodsbooks.com
gotwfmaybe a www. in there
faenilworks ;)
gotwfinvest a few hours there and you can learn a lot.
djsmy recent interest in zfs on luks was prompted by a drive failing... would be nice to get a replacement
djsbut I didn't like the idea of sending it back with any personal/sensitive data on it
PMTdjs: i mean, how badly did it fail? you can still zero it if it's just some bad sectors.
djsPMT: well, I put in a replacement but left the bad disk in my mirror and after resilvering it seems to be perfectly fine again
djsso maybe it was just a loose cable or something
djsbut yeah, I was planning on overwriting the data if it continued to drop off the bus
djsgot me thinking that I should be encrypted though, just to make replacement easy
PMTdjs: encryption does mitigate you being concerned about people doing recovery on your disk, but it also mitigates your ability to recover your own disks
PMT(not claiming you should or shouldn't do it, just observing)
faenilgotwf: great, thanks
djshaha, true enough... I suspect zfs with compression probably mitigates against most people trying to do recovery on my disk in the first place
djsincluding me! ;)
faenilgotwf: unfortunately my weekend is almost finished, and I did not get anywhere :/
djszfs mirrors with zfs send/recv backups hopefully ensure I don't need to do recovery on my disks too
PMTi mean, zfs with compression isn't really that obscure or opaque for anyone seriously looking at your disk
faenilPMT: how do you protect backups?
PMTfaenil: is this a general question, or how I personally do it?
faenilthe latter
djsI realize that compression isn't protection but someone would have to be pretty serious before they would be able to pull data out of a zfs pool
faeniland doesn't encryption also mitigate problems in case of theft or anything of the likes?
faenilI mean, evil-maid is still unsolved, mostly, at least for home servers usecases
faenilbut simple "theft" should be fixed by luks
PMTfaenil: sure, i'm not saying it's a bad idea, just pointing out the tradeoffs.
faenilsure, I'm just curious to get more opinions :)
stratactPMT: is there any other missing ZFS features that Linux might be annoyed with from migrating to it?
djsdefinitely tradeoffs... mostly the downside is additional complexity and poorly documented setup
PMTi'm presently just using crashplan both locally on my storage server and with an offsite location, since it does not appear to be screwing up the crypto in an obvious way from looking at it. if I were more paranoid, I'd probably use something like tarsnap or an option that let me use e.g. a yubikey or similar for not being able to extract the key, but it hasn't been an issue so far.
PMTstratact: features missing from ZoL, or missing from ZFS in general?
faenilPMT: so you are using luks as well
PMTstratact: I miss having something like fmd, and if I used ZoL in a nontrivial configuration, I'd probably miss the TRIM support, though most SSDs work around that well enough these days that you only care so much
PMTfaenil: I don't think I mentioned LUKS anywhere.
faenilPMT: you mentioned crypto, so I went with a blind guess :)
PMTfaenil: no, i meant that crashplan's over the wire protocol and on-disk representation don't have any glaring flaws in how they store data. :)
faenilah ok :)
sveinseooi, is there any professional drive recovery services that offer zfs recovery? -- I suppose the whole idea of zfs is among many others, the ability to have built-in redundancy, so perhaps the need is not there.
stratactPMT: In my zpool I currently have besides multi_vdev_crash_dump: async_destroy, empty_bpobj, lz4_compress, spacemap_histogram, enabled_txg, hole_birth, extensible_dataset, embedded_data, bookmarks, filesystem_limits, large_blocks. Should any of these not be active?
GrayShadesveinse: you're always one mistaken zpool destroy command or zfs bug away from data loss
PMTsveinse: i knew of one person who wrote about their experiences doing it, but i imagine it's far from a ubiquitous service
PMTstratact: http://open-zfs.org/wiki/Feature_Flags is a useful reference, I haven't done much migration between platforms of late but it's been relatively painless for me in the past
sveinseI suppose any company that offers zfs recovery would have resources vested deep into the core details of zfs, so they would most probably be a part of the zfs community, right?
stratactThanks for the link, looks like I should be good, since ZoL has everything else besides multi_vdev_crash_dump, which I will leave inactive. :)
PMTsveinse: I've not personally had occasion to look into ZFS recovery services, so I can't speak from experience, just from happening upon someone writing about doing it in the past
sveinseSure, I'm just waving some thoughts
PMTI imagine that part of it might be that at the point where you have enough invested in ZFS to need recovery services, you probably also have a nonzero amount of inhouse investment
ryaoUsually, you would go to backups rather than ask someone to fix things if they get really bad. Needing someone to try recovering a pool is very rare.
gotwfryao: Do you have edit access to zfsonlinux github wiki?
ryaogotwf: I don't see anything along those lines in github.com. I could try going through git to be certain, but I am under the impression that the HTTPS interface supports editing.
gotwfryao: maybe I am brain dead but I missed it. Also not all that familiar with github. I wrote up a zol on voidlinux article that might be nice to get linked to in the "Getting Started" section https://github.com/zfsonlinux/zfs/wiki/Getting-Started
gotwfryao: My guide here http://www.teamcool.net/posts/zol-voidlinux.html
dasjoeHeh, "dig hostname" shows the new ip already, "ping hostname" pings the old one. Anything I can do?
gotwfI don't want to have to clone the wiki and issue a pull request just to add a link.
bunderdasjoe: you run bind on that machine? maybe its cached
gotwfryao: I am not a zol collaborator so I don't think I can edit wiki pages. Smart call else if public would be spammed to death. Hence I need to get a collaborator to add that link.
faenilmm I was just reading up on the fact that deduplication doesn't work if ZFS is on LUKS because the encrypted data is salted, but then how do snapshots work?
DHowettfaenil: deduplication looks for identical blocks; snapshots are lightweight maps pointing to existing blocks, iirc.
DHowetta snapshot doesn't copy the blocks, and it doesn't look for identical blocks
faenilDHowett: so snapshot will actually work, but deduplication won't right?
faenilok, thanks
Nukiendasjoe, Hrm, crap I can't find the --metadata=1.0 in my notes. I remember reading about needing it, but not why.
NukienThanks for the typo catch on raid-devices ...
Nukienaaaand ... I copied stuff from *all* over the intertubes ...
gotwfNukien: what bootloader are you using
gotwfif this is dmraid, then depending on bootloader, you may require "legacy" metadata
gotwfe.g. syslinux
gotwfwhy you'd be using dmraid on zol is beyond me. if i missed context, apologies
gotwfah, yeah, i see you are
Nukiengotwf, regular old stock grub
Nukiengrub is installed on all disks too
gotwfin wh/case, iirc, currently metadata is 1.2. I'm not sure about grub and mdraid. only ever used mdraid with syslinux, wh/i am pretty sure required 1.0 or earlier? Been a LONG time since I set that stuff up though. Best advice is to read man mdraid. Grab a sandwich. you'll be a while...
gotwfthat said, i seem to recall metadata version not being an issue with grub
gotwfbut... it has been a _long_ while so pls verify
NukienYeah I know. I seem to remember it was something to do with the efi booting though
gotwfuefi? grrrmbl..... don
gotwft get me started...
Nukiensoem google-fu ought to turn up the info
Nukienheheh, tell me.
gotwfarchwiki is pretty good w.r.t. uefi stuff.
gotwfas is rodsbooks
NukienYup, many sources used. There should be comments in script with URLs where I picked stuff up.
gotwfarch used to have a really good mdraid howto but the wiki has since been sliced and diced and stuff is more scattered so you may have to dig
ray13finaly have my reporting all good.
DeHackEdACTION is watching the mahrens presentation from earlier this year.... 2.6 hours long..
bundermdraid, what heresy is this
bunderDeHackEd: link?
DeHackEdbunder: https://www.youtube.com/watch?v=ptY6-K78McY
bunderwhy unlisted?
DeHackEdum... dunno. didn't notice
DeHackEdZIL bits are news to me, and interesting. explains what I suspected: just why the ZIL is latency-sensitive
faenilgotwf: setting bios_grub flag was enough to get grub-install to not fail...not on to actually make it boot :D
zfscolmbuckley opened issue zfsonlinux/zfs#4836 - USB resets/hangs with ZFS load <https://github.com/zfsonlinux/zfs/issues/4836>
bunderoh dear
bunderbuggy usb
bunderprobably not much we can do there heh
DeHackEdwell it's not ZFS's fault. the question is, who is at fault? likely the USB layer or the drive controller
zfscolmbuckley closed issue zfsonlinux/zfs#4836 - USB resets/hangs with ZFS load <https://github.com/zfsonlinux/zfs/issues/4836>
zfsRudd-O commented on pull request zfsonlinux/zfs#3560 - Refactor SYSV init, initramfs and dracut (Phase 3) by FransUrbo <https://github.com/zfsonlinux/zfs/pull/3560#discussion_r70190937>
zfsRudd-O commented on pull request zfsonlinux/zfs#3560 - Refactor SYSV init, initramfs and dracut (Phase 3) by FransUrbo <https://github.com/zfsonlinux/zfs/pull/3560#discussion_r70190978>
fearedblissdasjoe: I dont have extlinux in my guide anymore (at leastnot in the main zfs guide)
stratactfearedbliss: thank you for making zfs versions of sysrescue
fearedblissstratact: np :), glad you are finding them helpful
stratactI'm gonna need them to help me migrate back to Linux from FreeBSD while keeping my /home zpool
fearedblissstratact: why are you migrating away from freebsd?
stratactIt didn't work out as a desktop OS for me.
fearedblissstratact: freebsd is better for servers I think due to the hardware support
fearedblissstratact: what linux distro you gonna go with>?
fearedblissDeHackEd: ooh.. I dont think I've seen this before.. thanks for sharing
stratactI think FreeBSD could work to be a great desktop OS, I just think it needs more internet publicity and a larger userbase to help push motivation, like with Linux. But primarily I think the developers mostly care about server, therefore lack of interest for desktop. Not to say it can't be used as a desktop as of now, it's just too limiting for me.
stratactWell anyway it helped me appreciate ZFS more :)
sponixstratact: Check into the PC-BSD project.. They share your vision of brining FreeBSD to the Desktop
djsso there was a comment earlier about dedup not working with zfs on luks which sounds completely off to me
djsmy expectation would be that zfs is deduping based on its hashes of the blocks, not the underlying data in the block layer
djsanyone know for sure? I'm not using dedup but that just has me scratching my head too hard
FireSnakespeaking of freebsd, sorry for repeating myself: can anyone try to reproduce #4832 either on linux or freebsd?
zfs[zfs] #4832 - reproductible bug: steps to trigger hang of a pool <https://github.com/zfsonlinux/zfs/issues/4832>
ptx0djs: they meant LUKS on ZVOL I think
djsptx0: ahh yeah, that would make more sense for sure
BoobuigiDeHackEd: If that lecture video taught me anything, it's that some people pronounce "fsck" as "eff ess see kay."
zfsdweeezil commented on commit zfsonlinux/zfs@ab9f4b0b82 - SIMD implementation of vdev_raidz generate and reconstruct routines <https://github.com/zfsonlinux/zfs/commit/ab9f4b0b824ab4cc64a4fa382c037f4154de12d6#commitcomment-18188920>
zfsdweeezil opened pull request zfsonlinux/zfs#4837 - Prevent null dereferences when accessing dbuf kstat by dweeezil <https://github.com/zfsonlinux/zfs/pull/4837>
ptx0Boobuigi: some say "fs check" or "filesystem check"
Boobuigiptx0: I might use the latter during a presentation but would probably just say "eff sock..." Regardless, the video opens with two really informative diagrams.
ptx0I used to say fisk before some customer thought I was telling her to go "fist her box"
ptx0we did end up losing the customer
ptx0I don't think it was a major loss.
bunderi've always used f s c k, i never liked to call it fuck or anything else that sounds weird
bunderthen again, that's how i say everything in linux, i call it e x t 3, not 'extended 3' etc
bunderf s tab, e t c, u s r, t m p, i'm weird heh
bundermahrens saying "i octal" for ioctl made me cringe a little
ptx0that's how most say it
bunderdunno i just think it just sounds weird considering it's something to do with IO, and nothing with octals
ptx0that's an odd thing to obsess over
bundernot pronouncing stuff right makes you look like an idiot to old unix fogies
chungypeople that pronounce "OS" as a single syllable... weirdos. :P
gotwfmaybe. but it is also no longer pc to pronounce tty as titty, wh/is how learned it from old dogs. damn near got fired for it and there wasn't even a woman present
gotwfheh.... just made me want to use the term moaarrrr!
gotwfseeing how somebody needs to be that hung up
gotwfooohhh. a titty hang...
gotwfi is so punny
gotwfanyways, yeah, i had manager saying "oz" ina meeting and it was halfway thru before i figured out they meant OS