15:02:02 <ydary> #startmeeting oVirt Weekly Sync 15:02:02 <ovirtbot> Meeting started Wed Feb 4 15:02:02 2015 UTC. The chair is ydary. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:02:02 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic. 15:02:18 <ydary> #topic Agenda and roll Call 15:02:19 <ydary> #info infra update 15:02:19 <ydary> #info 3.5.z updates 15:02:19 <ydary> #info 3.6.0 status 15:02:19 <ydary> #info conferences and workshops 15:02:20 <ydary> #info other topics 15:02:30 <ydary> who is here for weekly? 15:02:35 * sbonazzo here for integration and rel-eng 15:02:36 * bkp here 15:02:48 * bkp here for community 15:02:57 * lvernia is here. 15:03:56 <ydary> tal: eedri: dcaro|mtg: are you joining weekly? 15:04:31 <ydary> doron_afk: fabiand: are you joining weekly? 15:04:50 <ydary> mskrivanek: are you joining weekly? 15:05:04 <ydary> #topic infra update 15:05:10 <dcaro|mtg> hey 15:05:14 <dcaro|mtg> ydary: I'm here 15:05:35 <ydary> sbonazzo: dcaro|mtg: can you update on infra? 15:05:41 <dcaro|mtg> So this week we had a huge outage on phoenix lab 15:06:02 <fabiand> ydary, hey - I'm here, let's see if I stick here wthe whole time 15:06:47 <dcaro|mtg> related to the new networking range, combined with vdsm removing network configurations and an issue with NUMA and 3.4 vms running on 3.5 upgraded hosts 15:07:25 <dcaro|mtg> everything is up and running again, we have a hosted engine running 3.5, and all but one host running 3.5 15:07:37 <dcaro|mtg> (production and hosted engine DC are running 3.5) 15:08:00 <dcaro|mtg> jenkins dc is running on 3.4 compatibility 15:08:20 <dcaro|mtg> I'll try to finish the upgrade toda 15:08:54 <dcaro|mtg> no further outages are expected and the env should be already stable but there are a few issues still affecting it 15:09:03 <dcaro|mtg> (I have to open bugs for the yet) 15:09:28 <dcaro|mtg> * Migration is not working, sanlock complains about pid lock not existing and fails migration 15:10:05 <dcaro|mtg> * 3.4 VMS need to be reconfigured manually to change NUMA default no not preferred (even if NUMA is not being used) 15:10:07 <ydary> #info Outage this week on phoenix lab, that has been resolved, due to networking and upgrade issues. 15:10:15 <dcaro|mtg> libvirt refuses to run them ^ 15:10:39 <ydary> #info no further outages are expected and the environment should be already stable but there are a few issues still affecting it. 15:11:41 <dcaro|mtg> * and vdsm 3.5 failed to start in one host by an noKeyException on vdsm-restore-net-config:88, bonds[bond] did not exist when showing a log and made the vdsm startup fail 15:12:07 <ydary> #info issues to be opened: Migration is not working, 3.4 VMs need to have NUMA config, even when not used. 15:12:47 <ydary> #info another issue: vdsm 3.5 failed to start in one host by an noKeyException. 15:13:06 <dcaro|mtg> I'm also upgrading the hosts to fc20 (3 left), no downtime expected either. 15:13:42 <ydary> dcaro|mtg: upgrading what hosts for what purpose? 15:14:09 <dcaro|mtg> the srv01, srv03 and srv04 hosts to fc20 to allow nested virtualization 15:14:20 <dcaro|mtg> all the others have been already upgraded 15:14:43 <ydary> dcaro|mtg: this is for CI testing? 15:14:55 <dcaro|mtg> ydary: not necessarily, also node uses nested 15:15:56 <ydary> dcaro: this for the Jenkins workers? 15:16:31 <dcaro> ydary: yes, the jenkins slaves will run on top of those hosts 15:17:21 <ydary> #info In process to upgrade Jenkins hosts to fc20 to allow nested virtualization (srv01, srv03, srv04) 15:17:28 <ydary> dcaro: anything else? 15:18:34 <dcaro> ydary: except to have a bunch of failed jobs from the last two days 15:18:54 <ydary> dcaro: why? due to upgrades? 15:20:07 <dcaro> ydary: due to the downtime, the storage failed, what made the hosts fail, and all the vms fail 15:20:26 <dcaro> ydary: then vdsm messed the networks and the hosts lost connectivities, so the vms running also lost it 15:21:55 <ydary> #info Jobs failures from the last two days are expected to chain failure that started in network\storage that caused environment downtime. 15:21:56 <dcaro> ydary: all that while the engine refused to start, and prolonged the outage 15:23:33 <ydary> #action dcaro Will open bugs on issue he encountered in automation environment upgrade. 15:23:56 <ydary> dcaro: Anything else this week? 15:25:02 <dcaro> ydary: nop 15:25:08 <ydary> dcaro: thanks! 15:25:25 <ydary> sbonazzo: anything to add? 15:25:32 <sbonazzo> ydary: nothing 15:25:46 <ydary> moving on to 3.5.z 15:25:50 <ydary> #topic 3.5.z updates 15:26:04 <ydary> sbonazzo: you want to update on integration? 15:26:17 <sbonazzo> sent status this morning here: http://lists.ovirt.org/pipermail/users/2015-February/031116.html 15:26:32 <sbonazzo> bottom line, no progress since last week, just 1 bug less 15:26:49 <ydary> #info 3.5.z status update: http://lists.ovirt.org/pipermail/users/2015-February/031116.html 15:27:15 <ydary> sbonazzo: any commits worth mentioning? 15:27:36 <ydary> anyone has something to add to changes in 3.5 branch? 15:28:13 <sbonazzo> ydary: not on integration side 15:29:23 <ydary> lvernia: doron_afk: tal: fabiand: awels: anyone has something to add to changes in 3.5 branch? 15:29:37 <ydary> mskrivanek: ^ 15:29:40 <awels> ydary: Not me 15:29:41 <fabiand> ydary, no, I think for now we are good from Node 15:29:51 <lvernia> ydary: No. 15:30:11 <ydary> moving on to 3.6.0 updates 15:30:13 <ydary> #topic 3.6 status 15:30:33 <ydary> sbonazzo: any update on the release side? 15:30:35 <sbonazzo> sent status this morning here: http://lists.ovirt.org/pipermail/users/2015-February/031117.html 15:30:45 <sbonazzo> bug count is still increasing 15:30:54 <ydary> #info oVirt 3.6.0 status: http://lists.ovirt.org/pipermail/users/2015-February/031117.html 15:31:00 <sbonazzo> action item from last week was to report about 3.6 compatibility support on nightly 15:31:12 * lvernia listens carefully. 15:31:14 <ydary> #info Bug count increased from last week. 15:31:38 <sbonazzo> vdsm is not yet supporting it, missing a patch http://gerrit.ovirt.org/37384 delayed waiting libvirt support in el7.1 15:31:58 <sbonazzo> so for having 3.6 compatibility support we need to wait for RHEL / CentOS 7.1 15:32:04 <lvernia> ? 15:32:15 <ydary> sbonazzo: what is missing from CentOS? 15:32:31 <sbonazzo> Fedora 20 won't receive libvirt updates so it won't support 3.6 15:32:41 <lvernia> Why not have it reported as supported as long as the right version of libvirt is installed? 15:32:51 <lvernia> It shouldn't be blocked on any RHEL/CentOS release... 15:33:01 <lvernia> Just not work for now on RHEL/CentOS. 15:33:07 <sbonazzo> ydary: CentOS 7.0 need a new libvirt version providing VIR_MIGRATE_AUTO_CONVERGE support 15:33:37 <ydary> sbonazzo: what is the impact of this parameter missing? 15:33:54 <sbonazzo> lvernia: you should convince danken to merge the patch :-) 15:34:02 <lvernia> Ugh. 15:34:17 <ydary> sbonazzo: 3.6.0 is not a stable branch, some things can not work and it can still be ok 15:34:23 <sbonazzo> ydary: not really sure since I'm not in virt team, but as far as I understand it's required by vdsm for enabling 3.6 support 15:34:35 <lvernia> +1 :) 15:34:43 <ydary> mskrivanek: can you comment? ^^ 15:35:14 <ydary> anyone here can comment for virt? 15:35:15 <sbonazzo> ydary: just to close the support review, RHEL / CentOS 6 won't have 3.6 support by design 15:35:39 <sbonazzo> ydary: Fedora 21 will receive new libvirt from virt-preview repo 15:35:57 <sbonazzo> ydary: Fedora 22 probably won't be able to run vdsm since it will move from python2 to python3 15:36:18 <sbonazzo> ydary: and Fedora 20 is going EOL this june 15:37:27 <ydary> sbonazzo: this sounds like a mess.. so when will be able to use 3.6 then? test new features and so on? 15:37:37 <ydary> sbonazzo: and on what platforms? 15:37:58 <mskrivanek> aah 15:38:19 <mskrivanek> sbonazzo: yeah, correct 15:38:19 <sbonazzo> ydary: from my point of view, Fedora 21 and EL 7.1 will be able to support vdsm with 3.6. not sure about ETA for EL 7.1 15:38:41 <sbonazzo> ydary: Fedora 20 and EL6 will have only 3.5 support 15:38:54 <mskrivanek> impact of what? it does support it, it's just that it reports the 3.5 compatibility only. so you want be able to use 3.6 features. but you can always run 3.6 vdsm:) 15:38:56 <ydary> sbonazzo: didn't we decide to drop the fc21 support for 3.6? 15:39:40 <sbonazzo> ydary: yes, but vdsm is not only ovirt, it's also Fedora package. so vdsm team can build and release it for Fedora 21 within fedora repositories 15:39:47 <mskrivanek> ydary: why do we care about Fedora in context of 3.6/libvirt? 15:40:01 <ydary> mskrivanek: the entire point is to try out new features prior to release 15:40:03 <mskrivanek> ydary: F20 virt-preview does have everywhere needed for 3.6 at the moment 15:40:30 <sbonazzo> mskrivanek: no, as far as I know libvirt 1.2.12 is needed and f20 virt preview only has 1.2.9 15:40:31 <mskrivanek> ydary: plain F20 not. But that was/is the case since like, what, F18? We always needed livbirt/qemu from virt-preview.... 15:40:32 <ydary> mskrivanek: fc20 is EOL soon 15:41:01 <mskrivanek> sbonazzo: fromani assures me 1.2.9 is needed. Does it not work for you on 1.2.9? 15:41:20 <sbonazzo> mskrivanek: looks like you and me got different version requirements 15:42:04 <fromani> mskrivanek: sbonazzo: indeed. Can't think of anything that requires more than 1.2.9 (1.2.3 could be enough actually), especially for clusterLevel 3.6 15:42:29 <sbonazzo> fromani: so please comment on http://gerrit.ovirt.org/37384 15:42:34 <ydary> sbonazzo: We need to decide on how 3.6.0 will be tested and on what platform, can you sent a email to users lists to start this discussion? 15:42:38 <fromani> sbonazzo: sure thing 15:42:50 <ydary> sbonazzo: send* 15:43:19 <sbonazzo> ydary: ok 15:44:22 <ydary> #info Testing platform support for 3.6.0 is not clear. Each version has it's own issues and we want to let users play with new features. Discussion will be started on user list to resolve this. 15:44:45 <mskrivanek> sbonazzo: ok, i did 15:45:04 <sbonazzo> another point for 3.6, since 3.6 compat level won't be enabled on el6, hosted engine for 3.6 won't be released for EL6. documentation will be provided for migrating from el6 3.5 to el7 3.6 15:45:24 <ydary> #action sbonazzo Will start discussion on users list on what platforms we will enable 3.6.0 features and when. 15:46:33 <ydary> #info We remind that 3.6.0 will not support el6 hosts. 15:47:09 <ydary> sbonazzo: anything else? 15:47:23 <sbonazzo> ydary: that should be all on integration / rel-eng 15:47:46 <ydary> anyone can cover for infra updates? 15:48:53 <ydary> emesika: can you cover for infra? 15:49:04 <ydary> emesika: updates on 3.6.0? 15:49:08 <fromani> sbonazzo: done 15:49:56 <ydary> #info infra updates 3.6.0: no one attended from the team. 15:50:06 <ydary> lvernia: can you update on network? 15:50:14 <sbonazzo> fromani: replied to your comment 15:50:20 <lvernia> ydary: Yes, not much to update. 15:50:39 <lvernia> ydary: Solved a couple of bugs concerning the already merged features, QoS and MAC pool per DC. 15:50:49 <lvernia> ydary: Otherwise progress on other features as usual. 15:51:33 <lvernia> ydary: Also some effort debugging of 3.5 issues, but unrelated to 3.6 of course. 15:51:41 <lvernia> ydary: That's all for this week. 15:52:04 <ydary> #info Network updates for 3.6.0: Features review\development in progress, a few bugs solved on already merged features (QoS and MAC pool per DC). 15:52:30 <ydary> lvernia: thanks! 15:52:33 <fromani> sbonazzo: yes, el7.0 is out of luck (also noted on gerrit). 15:52:41 <lvernia> ydary: No problem. 15:52:43 <ydary> fabiand: any updates on node? 15:54:23 <ydary> tal: can you update on storage 3.6.0 work status? 15:56:46 <tal> ydary: Work in progress on the SPM removal (start of the POC), continuation of work on live merge and on CEPH intergration through Glance 15:56:50 <tal> Cinder* 15:57:56 <ydary> #info Storage updates for 3.6.0: Work in progress on the SPM removal (start of the POC), continuation of work on live merge and on CEPH integration through Cinder. 15:58:07 <ydary> tal: anything else? 15:58:16 <tal> No 15:58:26 <ydary> tal: thanks! 15:58:35 <ydary> awels: can you update on UX? 15:59:05 <awels> ydary: Sure, not much to update, making progress, tooltips are under review, rest in progress. 15:59:25 <ydary> awels: what is the rest? 16:00:06 <awels> ydary: Its in the spreadsheet, but main thing is REST api integration. 16:00:44 <ydary> #info UX updates for 3.6.0: Patternfly tooltips are under review, REST api integration in progress. 16:00:55 <ydary> awels: anything else worth noting? 16:01:05 <awels> ydary: Nope, that is it 16:01:12 <ydary> awels: thanks! 16:01:20 <ydary> mskrivanek: can you update on virt? 16:03:43 <mskrivanek> ydary: not much, crazy week with devconf, whole team in mtgs.... 16:03:44 <ydary> anyone here can update for virt? 16:04:27 <ydary> #info Virt updates for 3.6.0: no progress from last week. 16:04:43 <ydary> fabiand: are you back? can you update on node? 16:05:26 <ydary> #info Node 3.6 updates: no one attended from the team. 16:05:39 <ydary> doron_afk: can you update on SLA? 16:06:01 <ydary> anyone can update for SLA? 16:06:28 <ydary> #info SLA 3.6 updates: no one attended from the team. 16:06:49 <ydary> moving on to conferences and workshops 16:06:55 <ydary> #topic conferences and workshops 16:07:11 <bkp> Here. 16:07:13 <ydary> bkp: can you update on conferences and workshops? 16:07:24 <sbonazzo> ydary: oVirt application to 11th Intl. Conf. on Open Source Systems has been accepted (http://www.oss2015.org/) 16:07:28 <bkp> FOSDEM was a huge success, in terms of booth attendance, Virt/iaaS DevRoom attendance and feedback. It was also noted by other Red Hat projects how helpful and professional oVirt community members were in general. 16:07:50 <bkp> The half-day workshop for FOSSAsia in Singapore will be formally announced today. I spoke with FOSSAsia organizers at FOSDEM and they have registration all ready to go for us. 16:08:04 <ydary> #info oVirt application to 11th Intl. Conf. on Open Source Systems has been accepted (http://www.oss2015.org/). 16:08:17 <bkp> The FOSSAsia schedule (http://www.ovirt.org/Asia_Workshop_Mar_2015) is getting set. We have two open slots yet. One may be filled by Victor Ko, but we're settling on the topic. If anyone else wants to attend and give a talk, and help with the tutorail and overview sessions, you need to let me know very soon so we can get travel organized. 16:08:25 <ydary> #info FOSDEM was a huge success, in terms of booth attendance, Virt/iaaS DevRoom attendance and feedback. It was also noted by other Red Hat projects how helpful and professional oVirt community members were in general. 16:08:27 <bkp> *tutorial 16:08:43 <ydary> #info The half-day workshop for FOSSAsia in Singapore will be formally announced today. I spoke with FOSSAsia organizers at FOSDEM and they have registration all ready to go for us. 16:09:03 <ydary> #info The FOSSAsia schedule (http://www.ovirt.org/Asia_Workshop_Mar_2015) is getting set. We have two open slots yet. 16:09:35 <bkp> That's it from me 16:09:48 <ydary> #info One may be filled by Victor Ko, but we're settling on the topic. If anyone else wants to attend and give a talk, and help with the tutorial and overview sessions, you need to let bkp know very soon so we can get travel organized. 16:11:03 <ydary> #action If you want to give a talk in FOSSAsia for oVirt, there is a open slot, please reach out to bkp. 16:11:25 <ydary> #topic other topics 16:11:31 <ydary> Does anyone want to raise an issue? discuss something? 16:11:42 <bkp> I have some stuff... 16:11:54 <bkp> The moVirt subproject (http://www.ovirt.org/Project_moVirt) is now officially up for its incubator vote. oVirt Board list members are encouraged to vote yes or no for the incubator status by Feb. 18, 2015 (http://lists.ovirt.org/pipermail/board/2015-February/001151.html). 16:12:12 <ydary> #info The moVirt subproject (http://www.ovirt.org/Project_moVirt) is now officially up for its incubator vote. oVirt Board list members are encouraged to vote yes or no for the incubator status by Feb. 18, 2015 (http://lists.ovirt.org/pipermail/board/2015-February/001151.html). 16:12:23 <bkp> A mailing list for moVirt has been requested, and the project can be formally announced after its status has been established. 16:12:39 <ydary> #info A mailing list for moVirt has been requested, and the project can be formally announced after its status has been established. 16:12:58 <bkp> In other news, I met with key design and documentation team members during FOSDEM, and will be generating a formal proposal on re-designing oVirt's documentation process. Anyone interested in assisting with documentation, in terms of planning and content creation, should contact me. 16:13:18 <bkp> And finally, jbrooks has successfully tested an oVirt environment within the Ravello platform that can be used as a "TryVirt" for workshop attendees... something that they can sign into for one or two hours cumulative after a workshop to really play with oVirt in a multi-host online environment. 16:13:41 <ydary> #action bkp met with key design and documentation team members during FOSDEM, and will be generating a formal proposal on re-designing oVirt's documentation process. Anyone interested in assisting with documentation, in terms of planning and content creation, should contact him. 16:14:37 <bkp> That's all from me. 16:14:39 <ydary> #info jbrooks has successfully tested an oVirt environment within the Ravello platform that can be used as a "TryVirt" for workshop attendees. Users can sign into for one or two hours cumulative after a workshop to really play with oVirt in a multi-host online environment. 16:14:59 <ydary> any else has something to note? 16:17:00 <mskrivanek> go moVirt! 16:17:14 <ydary> mskrivanek: thanks :] 16:17:28 <ydary> so I guess that is it for today 16:17:34 <ydary> thanks for attending 16:17:41 <ydary> #endmeeting