14:07:08 <ewoud> #startmeeting infra weekly
14:07:08 <ovirtbot> Meeting started Mon May 13 14:07:08 2013 UTC.  The chair is ewoud. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:07:08 <ovirtbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
14:07:13 <ewoud> #chair eedri
14:07:13 <ovirtbot> Current chairs: eedri ewoud
14:07:21 <eedri> dcaro, ?
14:07:24 <eedri> knesenko, ?
14:08:00 * dcaro is here
14:08:16 <ewoud> #chair dcaro
14:08:16 <ovirtbot> Current chairs: dcaro eedri ewoud
14:08:22 <ewoud> agenda from http://www.ovirt.org/Infrastructure_team_meetings#2013-05-13
14:08:31 <ewoud> Introductions
14:08:32 <ewoud> Review of action items
14:08:32 <ewoud> Hosting
14:08:32 <ewoud> rackspace01 installed?
14:08:32 <ewoud> Puppet
14:08:34 <ewoud> Jenkins
14:08:36 <ewoud> Other business?
14:08:39 <ewoud> Trac review
14:09:07 * knesenko here
14:09:13 <ewoud> if that is all, I'd like to move to introductions and add team changes to the subject
14:09:17 <ewoud> #chair knesenko
14:09:17 <ovirtbot> Current chairs: dcaro eedri ewoud knesenko
14:09:29 <ewoud> #topic introductions and team changes
14:09:43 <ewoud> I think knesenko was approved as member
14:09:54 <knesenko> ewoud: :)
14:10:07 <eedri> knesenko, welcome :)
14:10:10 <ewoud> at least, I saw a few +1 and no -1
14:10:30 <eedri> knesenko, another infra member that will surely help and improve the ovirt project infra
14:10:41 <ewoud> #agreed knesenko is now an official infra member
14:11:03 <ewoud> #info eedri knows another potential infra member
14:11:28 <ewoud> knesenko: welcome :)
14:11:42 <ewoud> I think that's all for this topic
14:11:45 <knesenko> thank you all !
14:12:05 <ewoud> #topic Review of action items
14:12:17 <ewoud> eedri to send a vote request to list on making knesenko infra memeber for administrating jenkins,foreman,puppet
14:12:20 <ewoud> eedri document the new infra member process as it goes on
14:12:23 <ewoud> quaid look for compatible times for a docs sprint
14:12:25 <ewoud> dcaro eedri install the rackspace servers
14:12:28 <ewoud> Rydekull Add the jenkins slaves on a list on the wiki
14:12:30 <ewoud> ewoud restart the puppet ML thread
14:12:33 <ewoud> ewoud look into running piwik on openshift
14:12:35 <ewoud> knesenko talk to eedri about making work more transparant
14:12:38 <ewoud> knesenko handle jenkins backup and add script to jenkins git repo
14:12:40 <ewoud> ewoud install foreman.ovirt.org
14:12:43 <ewoud> dneary set up piwik
14:13:02 <ewoud> I did install foreman.ovirt.org with the help of knesenko, more about that on the topic puppet
14:13:32 <ewoud> eedri sent a vote request so that's done
14:13:55 <ewoud> I saw some discussion about the new member process, so done as well I think
14:14:09 <ewoud> we'll get to rackspace under hosting
14:14:48 <ewoud> piwik is already a trac ticket, so not re-adding
14:15:06 <ewoud> #action Rydekull Add the jenkins slaves on a list on the wiki
14:15:18 <ewoud> #action quaid look for compatible times for a docs sprint
14:15:21 <ewoud> #topic Hosting
14:15:29 <ewoud> eedri: dcaro any luck with rackspace?
14:15:44 <eedri> ewoud, i had some progress,
14:15:51 <eedri> ewoud, but was stuck on vpn acess
14:16:10 <eedri> ewoud, rackspace needs to add me & david to the vpn access, and it seems we're out of licenses
14:16:26 <eedri> ewoud, so that needs to be resolved before we can continue to install the servers
14:17:12 <ewoud> eedri: annoying
14:17:24 <eedri> ewoud, very
14:17:45 <eedri> ewoud, i'm thinking maybe start running jenkins slaves on alterway in the meantime
14:17:51 <eedri> ewoud, what do you think?
14:18:36 <knesenko> eedri: what do you mean  ? on ovirt-engine ?
14:18:47 <eedri> knesenko, yes.
14:18:57 <eedri> knesenko, the original plan was to install 'service' vm on alterway servers
14:18:59 <knesenko> eedri: do we have enough resources there ?
14:19:05 <eedri> knesenko, and jenkins slaves on rackspace
14:19:05 <ewoud> eedri: I think the servers lack memory
14:19:19 <eedri> ewoud, how many vms we'll be able to run there?
14:20:35 <ewoud> eedri: if you see that my slave dips into swap when building ovirt-engine, I think you need at least 8G to run those jobs and the hosts have 16G so I think 1 virtual slave?
14:21:17 <eedri> ewoud, hmm...
14:21:24 <eedri> ewoud, how about normal slaves?
14:22:20 <ewoud> eedri: maybe you can run a virtual slave on both servers?
14:22:46 <ewoud> the master still has plenty of memory left
14:23:06 <eedri> ewoud, i'm think we can implement what we said on the jenkins master
14:23:21 <eedri> ewoud, once we run out of resouces on alterway02, we can migrate jenkins to a vm
14:23:32 <eedri> ewoud, and free alterway01 as another host for ovirt-engine
14:23:34 <ewoud> eedri: but you want to run slave jobs on the master?
14:23:40 <eedri> ewoud, no i don't
14:23:52 <eedri> ewoud, it's bad practise and security risky
14:23:55 <ewoud> eedri: oh ok, so you want to move jenkins master into a VM now?
14:24:10 <eedri> ewoud, we said we'll do it once we have ovirt-engine running
14:24:26 <eedri> ewoud, and we just migrated jenkins to a vm in $company, works fine
14:24:33 <ewoud> eedri: ok
14:24:36 <eedri> ewoud, i'm not saying now
14:25:01 <eedri> ewoud, i'm saying once we see we are running out of resources on ovirt-engine
14:25:10 <eedri> ewoud, maybe after migrating resources.ovirt.org..
14:25:10 <ewoud> eedri: I think alterway02 needs some tuning with ovirt, but in general it already works and we installed foreman.ovirt.org on it
14:25:53 <eedri> ewoud, ok, so once we feel comferable with ovirt-engine we'll do it
14:26:07 <ewoud> eedri: and it's not possible to revoke quaids VPN access while you install it?
14:26:12 <knesenko> ewoud: eedri I think that we need some map/wiki of our resources ... because I am confused a little bit with all these names
14:26:37 <ewoud> knesenko: Rydekull wrote a great start for a naming convention, but we never got to finish it
14:27:10 <knesenko> ewoud: do you have a link ?
14:27:12 <ewoud> http://lists.ovirt.org/pipermail/infra/2013-April/002545.html
14:27:17 * ewoud was searching for it
14:27:20 <eedri> knesenko, we had a pad with all the info i think
14:27:36 * eedri thinks we should create a wiki with all the pads
14:27:50 <dcaro> I suppose that kiril is not talking only about the names, but also what do we have
14:28:02 <knesenko> dcaro: yes ...
14:28:26 <knesenko> dcaro: I am confused .... which resources do we have ... how many bare metal and vms ... etc
14:28:41 <ewoud> knesenko: yes, some inventory may be nice
14:28:56 <ewoud> knesenko: how many bare metal and vms is something foreman could start to answer for us
14:29:05 <eedri> knesenko, found it http://etherpad.ovirt.org/p/new_hosting_design_Jan_2013
14:29:25 <eedri> knesenko, this was the plan, not everything exists yet
14:29:47 <knesenko> eedri: we wanted to install nagios right ?
14:29:51 <knesenko> obasan: ^^
14:30:02 <eedri> knesenko, if we have resources i think it will help us
14:30:13 <knesenko> eedri: nagios can help us to monitor + understand what do we have today
14:30:27 <eedri> knesenko, meaning if it won't block migrating important service like gerrit/resources
14:30:35 <ewoud> knesenko: I was thinking about installing puppetdb on foreman as well and use stored configs to build a monitoring.ovirt.org
14:30:42 <ewoud> add munin to it for easy trending
14:30:42 <eedri> knesenko, i also think it doesn't nessasaritly requires a dedicated vm
14:30:53 <eedri> but we really need some monitoring system
14:31:01 <eedri> too many downtimes/out of space issues
14:31:03 <ewoud> eedri: I'd like a monitoring VM
14:31:24 <dcaro> +1
14:31:26 <obasan> eedri, ewoud I heard good stuff about icinga
14:31:32 <knesenko> ewoud: you mean a seprate machine that will run a monitoring service ?
14:31:38 <eedri> ewoud, that would be optimal, need to see if we have the resources
14:32:30 <snmishra> mpastern: around?
14:32:38 <ewoud> knesenko: monitoring and trending
14:32:51 <ewoud> eedri: we could start using alterway02 now
14:33:21 <ewoud> obasan: I did too, but had trouble finding packages in epel but maybe rpmforge isn't as bad as it used to be
14:33:57 <eedri> ewoud, what is our next vm after foreman?
14:34:03 <eedri> ewoud, that is critical for us to migrate
14:35:19 <ewoud> eedri: I think resources.ovirt.org is the easiest (besides maybe ovirtbot)
14:35:48 <eedri> ewoud, what does it contain? other than the releases & nightly repos
14:36:27 <ewoud> eedri: minutes from our meetings
14:36:42 <eedri> ewoud, ok
14:36:54 <eedri> ewoud, so a file server basiacally
14:37:01 <eedri> ewoud, no running service or something
14:37:10 <ewoud> eedri: pretty much
14:37:27 <ewoud> it uses apache to serve the files over HTTP, but that's it
14:37:31 <eedri> ewoud, just need to check how much space we need
14:37:35 <eedri> ewoud, and create the vm
14:38:01 <ewoud> eedri: yes, I installed the foreman-ovirt compute resource so you should be able to provision the VM from foreman
14:38:14 <ewoud> except there was something with the network I couldn't figure out in the few seconds I tested
14:38:19 <eedri> ewoud, nice
14:38:30 <eedri> ewoud, did you install foreman-proxy as well?
14:38:59 <ewoud> eedri: yes, DHCP gives out the IP but I think TFTP had some issue
14:41:46 <ewoud> so a short summary
14:41:57 <ewoud> we want to migrate resources.ovirt.org first
14:42:16 <ewoud> to do that we need to check how much space is needed
14:42:41 <ewoud> we want to install a monitoring VM with nagios or icinga and munin
14:42:51 <ewoud> correct?
14:42:59 <knesenko> seems ok
14:43:14 <eedri> yea, just add the create vm using foreman
14:43:22 <eedri> i'm sorry guys, i have to leave
14:43:28 <eedri> i'll catch up on the list
14:43:43 <knesenko> eedri: bye
14:43:45 <ewoud> #agreed we want to install a monitoring VM with nagios or icinga and munin on alterway02 using foreman
14:43:51 <ewoud> eedri: ok
14:44:08 <eedri> ewoud, just a small not (off topic), about publishing nightly repos.
14:44:16 <eedri> ewoud, i think it should be OK now, i sent email on it
14:44:18 <obasan> ewoud, icinga doesn't monitor network? only with munin/graphite?
14:44:20 <ewoud> #agreed we want to migrate resources.ovirt.org into a VM on alterway02 using foreman
14:44:43 <ewoud> obasan: I think icinga is pretty much a fork of nagios, but added some features such as IPv6
14:44:50 <ewoud> eedri: ok
14:45:13 <eedri> thanks, bye for now
14:45:13 <ewoud> #info we seem to be out of licences for the rackspace VPN
14:45:30 <ewoud> I'd really like to resolve the rackspace because it's a waste of resources having them idle
14:46:49 <dcaro> yep, I have the same issue there
14:47:04 <ewoud> obasan: would you like to look into installing icinga? maybe install it on a test VM of your own first?
14:47:34 <ewoud> dcaro: do you know if it's possible to revoke quaid's access and have you install it?
14:47:35 <obasan> ewoud, yes
14:47:43 <obasan> ewoud, I have no problem doing that
14:48:04 <dcaro> ewoud: I don't know, maybe calling will help xd
14:48:14 <ewoud> obasan: cool
14:48:22 <ewoud> #action obasan look into icinga for monitoring
14:51:05 <ewoud> I'm willing to look into installing VMs using foreman, but I'm away for DjangoCon the rest of the week
14:52:15 <dcaro> ewoud: I've successfully set up foreman to use rhevm as a compute resource
14:52:22 <dcaro> to create vms
14:52:36 <dcaro> if you need help just ask
14:52:47 <ewoud> dcaro: I already added the foreman-ovirt plugin and creating the VM works, just the full stack needs to be completed
14:52:54 <ewoud> so checking TFTP works etc
14:53:24 <dcaro> ok, yes, setted up that too
14:53:40 <ewoud> dcaro: then let's verify after the meeting you can log in
14:53:43 <karim> ewoud, happy is cool to do that actually
14:54:00 <dcaro> ewoud: ok
14:54:11 <ewoud> #action dcaro ensure VM can be created in foreman using ovirt on alterway02
14:54:36 <ewoud> karim: ?
14:55:03 <ewoud> we're running a bit low on time, so let's move on to the topic puppet
14:55:26 <ewoud> #topic puppet
14:56:04 <ewoud> #info foreman.ovirt.org installed by knesenko and ewoud
14:56:17 <karim> ewoud, oh men, i m tiredd.... i meant api is a cool way  to create vms in foreman
14:56:37 <ewoud> karim: very :)
14:57:07 <ewoud> there is a git repository on puppet@foreman.ovirt.org:puppet, but there are no SSH keys in place nor any files
14:57:19 <dcaro> xd
14:57:48 <ewoud> I'd like to start by adding users with their SSH keys and adding all existing hosts to puppet to see how it holds
15:00:23 <dcaro> nice, need help with the manifests?
15:00:44 <ewoud> long ago I wrote https://github.com/ekohl/ovirt-infra-puppet which I'd like to use as the basis now
15:02:45 <ewoud> dcaro: I'm thinking of making ovirt_infra::user the define to manage the ovirt infra members and create ovirt_infra::user::dcaro as a concrete class
15:03:01 <ewoud> then I'm not sure if we should just add all the users to host groups in foreman or specific hosts
15:03:58 <dcaro> wouldn't it be better to declate ovirt_infra::user as a class and just declare instances of it with different names?
15:04:11 <dcaro> we could even use hiera to get the users list
15:04:30 <dcaro> (so no changes on the manifests if new users are added or old are deleted)
15:04:54 <ewoud> dcaro: I'd like to use hiera, especially since we can separate that from the public git, but no experience
15:05:06 <ewoud> dcaro: but I don't think you can use a class more than once, hence a define
15:05:41 <dcaro> you can do this ovirt_infra::user{ ['dcaro', 'ewoud']: bla => bla,}
15:05:56 <ewoud> dcaro: only with a define afaik
15:05:57 <dcaro> it will declare two instances of user with two different names
15:06:03 <ewoud> dcaro: that works?
15:06:22 <dcaro> mmm, not, sorry, you are right
15:07:06 <dcaro> that's only for types/defines xd
15:07:48 <ewoud> dcaro: this time I will make a proposal on the ML and maybe we can look at it this evening?
15:07:59 <dcaro> ok
15:08:07 <ewoud> #action ewoud write a design proposal for the puppet repository
15:09:35 <ewoud> I don't think we have time for jenkins and already discussed on hosting
15:09:43 <ewoud> so I'd like to close the meeting
15:10:13 <ewoud> going once
15:10:38 <ewoud> going twice
15:10:48 <ewoud> #endmeeting