14:04:03 #startmeeting infra weekly meeting 14:04:03 Meeting started Mon Aug 26 14:04:03 2013 UTC. The chair is ewoud. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:04:03 Useful Commands: #action #agreed #help #info #idea #link #topic. 14:04:08 #chair eedri_ 14:04:08 Current chairs: eedri_ ewoud 14:04:15 ping obasan dcaroest Rydekull 14:04:20 ewoud, hello! 14:04:22 * obasan here 14:04:25 #chair obasan 14:04:25 Current chairs: eedri_ ewoud obasan 14:04:51 * dcaroest is here too 14:04:55 #chair dcaroest 14:04:55 Current chairs: dcaroest eedri_ ewoud obasan 14:05:42 agenda: http://www.ovirt.org/Infrastructure_team_meetings#2013-08-26 14:05:53 minutes from last week: http://resources.ovirt.org/meetings/ovirt/2013/ovirt.2013-08-19-14.01.html 14:06:02 obasan, ? 14:06:07 ok 14:06:10 eedri_, here 14:06:13 #topic items from last week 14:06:28 obasan update jenkins-ovirt.org to the latest lts 14:06:35 o7 14:06:38 obasan: any progress on that? 14:06:40 #chair Rydekull 14:06:40 Current chairs: Rydekull dcaroest eedri_ ewoud obasan 14:06:44 ewoud, done. of course 14:06:50 obasan: very good :) 14:06:58 ewoud, Also another small update on my side 14:07:07 ewoud, the coverity job should now be fixed once and for all 14:07:07 (and today I dont have to leave, cannot guarantee I wont be disturbed though) :-) 14:07:16 ewoud, since a patch was merged upstream to fix it 14:07:40 ewoud, also update some crucial jenkins plugins 14:07:45 ewoud, updated* 14:07:53 obasan: what was the coverity issue again? 14:08:12 Rydekull: I doubt anyone can guarantee they won't be disturbed :) 14:08:14 ewoud, the problem was in some new java convention that coverity thought that was problematic 14:08:36 ewoud, so coverity thought that the project failed compiling 14:08:41 ewoud, while it actually did not. 14:08:49 obasan: ok, good that it's fixed upstream 14:09:07 * ewoud doesn't like keeping patches 14:09:11 knesenko install f18 + 2 centos 6.4 on rackspace2 14:09:20 anyone knows if knesenko had time for that? 14:09:42 ewoud, I don't know. 14:09:45 eedri_, do you know? 14:09:56 ewoud, i think centos vms are installed and running on jenkins 14:10:08 ewoud, but they might be missing rpms, so not sure we're using them yet 14:10:31 ewoud, i suggest migrating the current jobs that use rhell64 label to centos64 14:10:40 ewoud, and finding out what rpms are missing 14:10:49 eedri_: sounds good 14:11:01 #topic hosting 14:11:02 ewoud, don't think he installed f18 vms yet 14:11:20 I saw a mail about more storage for rackspace 14:13:11 anyone can describe to me how it's setup? 14:13:25 ewoud, i think david added the disks, not sure if he updated the fs (lvm?) 14:13:26 dcaroest, ? 14:13:38 in the minutes from last week I see '14:03:29 its a kind of external … :)' 14:14:13 does that mean each host now has another 2 TB disk in it? 14:14:35 The storages were added phisycally, but not logically to the host (meaning they were not partitioned or mounted) 14:16:17 ok, would it be a candidate to try gluster? 14:17:00 ewoud, i think it should be probably better than local storage 14:17:11 I'm not familiar with gluster setups of ovirt, but if it's possible it would be nice to have some ha machines there 14:18:01 especially if the self hosted ovirt feature is ready 14:18:08 anyone willing to pick that up? 14:18:53 gluster is very easy to setup fyi 14:19:54 and ovirt supports gluster, yes, however, there are some caveats, not sure how our datastores are configured today 14:20:06 Rydekull: do I hear a volunteer? 14:20:44 I have no access to the new hosts, so someone will have to spend more time setting my accounts up then it takes to setup gluster :-) 14:20:57 I have 0 experience with gluster 14:21:27 Rydekull: I think you having access to those hosts is a good thing by itself 14:21:38 Im glad you think that ;-) 14:22:05 But yeah, sure, I can partition disks and install gluster and configure that 14:22:15 If I get access, that is 14:22:41 * ewoud doesn't know how the access is set up 14:22:50 but we'll manage that after the meeting 14:23:04 so all in favor of setting up gluster? 14:23:35 eedri_: obasan dcaroest Rydekull ? 14:23:42 +1 14:23:54 +1 14:24:53 #action Rydekull set up gluster on rackspace hosts 14:24:56 will it affect current data? 14:25:08 eedri_: I'd assume that since it's a new storage domain, it shouldn't 14:25:09 meaning all vms that are currently running (vm19) 14:25:14 ewoud, ok 14:25:21 Rydekull: can you confirm that? 14:25:33 Confirmed, if it is like described above with the disks 14:25:39 ewoud, we also decided that once that storage is up and running we can migrate backups to it 14:25:46 No 14:25:49 No no no 14:26:11 Rydekull, we have 2 disks there 14:26:24 Rydekull, one should be raid 1 mirror - for backups 14:26:27 No 14:26:33 Rydekull, the other one is for jenkins slaves 14:26:34 That's not backup, that's diskredundancy 14:26:39 Backups are stored somewhere else 14:26:47 Rydekull: but rackspace hosts are just jenkins slaves 14:26:51 Hmm 14:26:55 Right 14:26:55 Rydekull, it is somewhere lese 14:27:00 Rydekull, from alterway 14:27:01 No vital data, ok 14:27:10 I give, store backups :-) 14:27:32 Rydekull, yes.. storing backups 14:28:00 Rydekull, from other machines like ovirt.org/foreman/jenkins/gerrit 14:28:18 Rydekull, what lies mostly on linode1 now 14:29:48 sorry, I was away for a moment 14:30:24 so... anything else? 14:37:44 * Rydekull doodles 14:41:17 Hi everyone 14:41:18 * ewoud was interrupted 14:41:34 dneary: hi 14:41:38 anything else on hosting? 14:42:07 ewoud, not that I can think of 14:42:11 dneary: hello 14:42:17 Still catching up - I see the storage issue is in the process of being resolved? 14:42:36 dneary: yes 14:42:40 yay! 14:42:43 #topic jenkins 14:43:12 #info updated to latest LTS by obasan, including some critical plugins 14:43:42 #info in the process of adding more f18 and centos6.4 slaves 14:44:32 anything else on jenkins? 14:44:45 ewoud, network function tests for vdsm was added 14:45:01 ewoud, need to convert it to per patch mode 14:45:18 eedri_: I saw there were concerns about it becoming unreachable 14:45:30 is that solved or something we should keep monitoring? 14:46:05 ewoud, not afaik, we need gvallarelli response 14:46:20 ewoud, if those tests might potentially take a jenkins slave off, it's a problem 14:46:40 #info network functional tests added, still some concern that may make jenkins slaves unreachable 14:47:29 eedri_: ewoud , under normal circumstances they should not, but during the creating of the jobs, the slave got disconnected a couple of times, meaning that if the tests fail, they can potentially take a slave offline 14:47:29 eedri_: I agree, and some dedicated mini jenkins slaves would be nice 14:47:57 I was looking at linux containers to see if they can be used for those tests easily 14:48:14 and I'd assume that vdsm needs a lot less memory for a slave 14:48:21 I was not able to set one up yet (only spent a couple of hours looking) 14:48:47 yes, has relatively low resource needs 14:48:57 ewoud, maybe now with more storage on rackspace we can create mini vms 14:48:57 ewoud: it should not happen. 14:48:58 but it needs networking exclusive for itself 14:49:38 ewoud, we'll need to poc running it per network patch and see if jenkins can handle the load 14:49:44 it also depends on how the test are written. 14:49:48 gvallarelli: should not, but how likely is it that if we have per patch jobs that one will take the slave with itself? 14:50:07 that's why I could not give 100% guarantee that the slave can be disconnected. 14:50:30 *can't 14:50:36 gvallarelli: ok 14:51:00 +1 to a poc 14:51:11 ewoud: at the moment we're porting and fixing tests, we should be in an enough stable condition soon. 14:51:51 eedri_: I'm not aware of any decision taken in regards of how to trigger the job in an automatic way. 14:52:09 gvallarelli: jenkins can be triggered on any new gerrit patch set 14:52:19 gvallarelli, i think this should network developer responsibility 14:52:38 gvallarelli, once you provide infra a way to differ network commit, we'll implement it on jenkins 14:53:00 eedri_: can't we use gerrit labels? 14:53:01 eedri_: for us the simplest thing would be to prefix for example the commit title with 'networking' 14:53:09 gvallarelli, so for example if it means to check the commit header ("network..") than we'll check that 14:53:15 +1 14:53:26 ewoud, gerrit labels? 14:54:10 since gerrit 2.6 you can create custom labels and I think someone suggested that 14:54:46 the problem with labels is that you'll need one per each test group, for example, one label called network, one called storage... etc 14:55:21 right now you can't add combo boxes or custom values for the labes (only +1, 0 -2 ...) 14:55:34 dcaroest: but I think putting it in the commit message is also messy 14:56:04 ewoud: we can oput it in the gerrit comment 14:56:15 but that will force the developer to create a comment 14:56:59 (not that it's bad, but it's not fully automated, labes solution also has the same problem) 14:57:56 dcaroest: true, but prefixing the commit message is also not automated 14:58:21 ewoud: it only needs the developer to add that message and push 14:58:42 the other require the developer to go to gerrit and add a comment/review 14:59:16 still, I'm unsure I'd prefix the subject since that line is already so short 14:59:17 ewoud, it is 14:59:20 ewoud, partically 14:59:30 ewoud, via the gerrit commit template 14:59:34 ewoud, in ovirt-engine today 14:59:43 ewoud: I would not prefix the subject, I'll put it on a tag in the last lines 14:59:54 as the Change-id and the Bug.-Url 15:00:09 dcaroest: I'd be fine with that 15:00:17 since that's convention on metadata 15:00:33 ewoud, but if we already have the prefix in place for commits? 15:00:36 ewoud, can't we use that? 15:01:00 what's the reason for that prefix? (just curious) 15:01:26 eedri_: if it's already there, fine but I wouldn't want to force it on people 15:01:41 what if a patch touches multiple sections? 15:02:43 the subject will grow too much xd 15:03:00 exactly, and it's already short 15:03:06 50 chars or something IIRC 15:03:58 ewoud, it's a problematic issue and i don't think we'll solve it here and now 15:04:06 eedri_: agreed on that 15:04:10 ewoud, this is why for networking sake, i asked gvallarelli to provide info 15:04:17 ewoud, since we only have now networkking tests 15:04:26 ok, we've gone a bit offtopic 15:04:30 anything else on jenkins? 15:04:32 ewoud, it can only come from dev side 15:05:55 eedri_: on vdsm side I'm aware only of our efforts 15:06:06 to provide tests that are runnable in a ci fashion. 15:06:17 So for now you can assume that's only networking functional tests 15:06:38 anyway we're recently receving contributions for functional tests on the virt side. 15:06:54 but I dunno how much time it will require for the virt team. 15:07:30 ewoud, there is an open ticket on fixing setup + upgrade job 15:07:42 ewoud, i would like it running cause i think it's very important to the project 15:07:54 ewoud, but didn't saw any volunteers to pick it up :/ 15:09:42 eedri_: can you link it? 15:09:51 ewoud, the ticket? 15:10:22 ewoud, https://fedorahosted.org/ovirt/ticket/73 15:12:01 eedri_: I still have the puppet and spamassasin tickets pending 15:12:34 Hmm 15:12:53 Oh, btw 15:13:00 the RH-ticket seem to have gone through 15:13:11 we now have a spf-record on ovirt.org 15:13:13 ovirt.org. 300 IN TXT "v=spf1 include:linode01.ovirt.org ~all" 15:13:25 Which hopefully will help us in not getting blocket at gmail 15:13:30 blocked* 15:13:35 * Rydekull will monitor it a bit 15:13:44 Rydekull: nice 15:14:03 eedri_: what should be done about that ticket #73? 15:15:09 ewoud, there is a job already, just need to fix the code for it - since setup was changed to otopi 15:15:16 ewoud, and it used to work on legacy setup 15:15:30 eedri_: looks like the answers.file was changed, so maybe we should ask someone who was involved with changing the installer to otopi? 15:15:41 Rydekull: it fails the test from kitterman tools 15:15:43 eedri_: it could indicate a regression in the code itself 15:15:49 http://www.kitterman.com/spf/validate.html 15:16:08 ewoud, i doubt it, since the whole infra was changed, including answer file 15:16:16 ewoud, so the job is not relevant anymore 15:16:28 ewoud, needs to be rewritten for new setup 15:16:57 eedri_: I wouldn't know how to test it 15:17:18 Rydekull: shouldn't it be a:linode01.ovirt.org or something like that? 15:21:44 ok, closing the meeting 15:21:46 #endmeeting