[Linux-PowerEdge] Mirror OMSA repository

Mike Hanby mhanby at uab.edu
Mon Feb 4 09:54:30 CST 2013


Howdy,

I had encountered an issue back in Oct 2011 where my clients were reporting failures accessing the local yum OMSA repo. I tracked it down to missing .htaccess symlinks.

I added the code to automatically generate the links following the repo sync. This was back in the OMSA 6.5.1 days, so perhaps it's not needed anymore. And possibly I was the only one experiencing the issue :-)

Here's the post were I reported the issue:
http://lists.us.dell.com/pipermail/linux-poweredge/2011-October/045401.html

From: David.Harding at brighthouse.co.uk [mailto:David.Harding at brighthouse.co.uk]
Sent: Monday, February 04, 2013 9:17 AM
To: jean-marc vizier
Cc: linux-poweredge at dell.com; linux-poweredge-bounces at dell.com; Mike Hanby
Subject: Re: [Linux-PowerEdge] Mirror OMSA repository

Jean-Marc, Mike (Hanby)

Thanks, both.
I have found that a combination of including both the System IDs (the system.ven_0x1028.dev value) *and* the Hardware names (ie. per300 etc) in the includes along with excluding the unnecessary architectures from the community repository yields the smallest repository footprint.
What I don't really know, is, do I have everything I actually need? I guess time will tell (unless anyone has a suggestion as to how to test that out without actually installing any/everything?).

I have a question for each of you.
Mike. In your script; what, exactly, is the purpose/benefit of linking the _tools/htaccess_repo_topdir and htaccess_tools files to .htaccess in the appropriate directories, please?
Jean-Marc, in your script; what is the purpose/benefit of editing the mirror server IP address into the mirrors_cgi script (into the serverName variable), please?

Here below please find the rsync that I have settled on for the set of servers:
  pe2950 (system.ven_0x1028.dev_0x01b2),
  per210 (system.ven_0x1028.dev_0x02a5),
  per300 (system.ven_0x1028.dev_0x020f),
  per410 (system.ven_0x1028.dev_0x028c),
  per510 (system.ven_0x1028.dev_0x02f1) and
  per720 (system.ven_0x1028.dev_0x048c)
on RHEL (actually Oracle Ent. Linux) 5, 64-bit
If anyone sees any issues with the selection, please let me know...(I could squish up the excludes for all OMSA 6 versions, as per the exclude of OMSA 5 versions..)

/usr/bin/rsync -azHv --delete-before --delete-excluded --stats --partial --timeout=160 \
   --exclude /community/deb \
   --exclude community/el4* \
   --exclude community/sles* \
   --exclude community/el5-i386 \
   --exclude community/el6-i386 \
   --exclude /dists/ \
   --exclude /hardware/vcenter-plugin \
   --exclude /hardware/OMSA_5* \
   --exclude /hardware/OMSA_6.0.* \
   --exclude /hardware/OMSA_6.1 \
   --exclude /hardware/OMSA_6.2 \
   --exclude /hardware/OMSA_6.3 \
   --exclude /hardware/OMSA_6.4 \
   --exclude /hardware/OMSA_6.5 \
   --exclude /hardware/OMSA_6.5.1 \
   --exclude /hardware/OMSA_6.5.2 \
   --exclude /hardware/OMSA_6.5.3 \
   --exclude /hardware/OMSA_7.0 \
   --include *dev_0x01b2* \
   --include *dev_0x02a5* \
   --include *dev_0x020f* \
   --include *dev_0x028c* \
   --include *dev_0x02f1* \
   --include *dev_0x048c* \
   --include pe2950 \
   --include per210 \
   --include per300 \
   --include per410 \
   --include per510 \
   --include per720 \
   --include /hardware/latest \
   --exclude *ven_0x1028.dev_* \
   --exclude *ven_0x1028_dev_* \
   --exclude *suse* \
   --exclude *rh30* \
   --exclude *rh40* \
   --exclude *rh50 \
   --exclude *rh60 \
   --exclude "pe*" \
   --exclude "dx60*" \
   --exclude *beta* \
   --exclude *testing* \
   rsync://linux.dell.com/repo /srv/yum/dell

Cheers,
dave












From:        "jean-marc vizier" <jvizier at gmail.com<mailto:jvizier at gmail.com>>
To:        <David.Harding at brighthouse.co.uk<mailto:David.Harding at brighthouse.co.uk>>,
Cc:        <linux-poweredge at dell.com<mailto:linux-poweredge at dell.com>>, <linux-poweredge-bounces at dell.com<mailto:linux-poweredge-bounces at dell.com>>, "Mike Hanby" <mhanby at uab.edu<mailto:mhanby at uab.edu>>
Date:        22/01/2013 14:08
Subject:        Re: [Linux-PowerEdge] Mirror OMSA repository
________________________________



Hello,

Yes it was quite the same rsync error.

You will find The script i made :
I clear the paths i use so change them before. The rsync is the last one i get. The dev link correspond to dell mapping. It allow to reduce the omsa version and hardware plateform. I cut the size by near 3.
Put the ip of your mirror server in this variable MIRROR_SERVER to allow the repo to work.

Hope it helps ;)

#!/bin/sh
#########

#   /_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
#  /_/          Vizier Jean-Marc      /_/
# /_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/


#-----------------------------------#
# 1955 system.ven_0x1028.dev_0x01bb #
# m610 system.ven_0x1028.dev_0x0287 #
# m620 system.ven_0x1028.dev_0x04db #
# r610 system.ven_0x1028.dev_0x0236 #
# r620 system.ven_0x1028.dev_0x04ce #
# r720 system.ven_0x1028.dev_0x048c #
#-----------------------------------#

#Mirror linux.dell.com<http://linux.dell.com/>

MIRROR_PATH=""
MIRROR_HOST=linux.dell.com<http://linux.dell.com/>
LOGS=""
DIST=dell
MIRROR_SERVER=""

DATE=`date +"%H%M%S"`
DATEF=`date +"%Y%m%d"`

mkdir -p ${LOGS}/${DATEF}

/usr/local/bin/rsync -a --delete-before --delete-excluded --progress --human-readable \
   --stats --archive --partial --timeout=160\
   --exclude hardware/OMSA_5* \
   --exclude hardware/OMSA_6.0.* \
   --exclude hardware/OMSA_6.1 \
   --exclude hardware/OMSA_6.2 \
   --exclude hardware/OMSA_6.3 \
   --exclude hardware/OMSA_6.4 \
   --exclude hardware/OMSA_6.5 \
   --exclude hardware/OMSA_6.5.1 \
   --exclude hardware/OMSA_6.5.2 \
   --include hardware/OMSA_6.5.3 \
   --include hardware/OMSA_7.0 \
   --include *dev_0x01bb* \
   --include *dev_0x0287* \
   --include *dev_0x04db* \
   --include *dev_0x0236* \
   --include *dev_0x04ce* \
   --include *dev_0x048c* \
   --exclude *ven_0x1028.dev_* \
   --exclude *ven_0x1028_dev_* \
   --include hardware/latest \
   --exclude *suse* \
   --exclude *rh30* \
   --exclude *rh40* \
   --exclude *rh50 \
   --exclude *beta* \
   --exclude *testing* \
   --exclude hardware/vcenter-plugin \
   rsync://linux.dell.com/repo<http://linux.dell.com/repo> $MIRROR_PATH \
| grep -v uptodate > ${LOGS}/${DATEF}/${DIST}-${DATE}.log 2>${LOGS}/${DATEF}/error_${DIST}-${DATE}.log

sed -i 's/os.environ.get("SERVER_NAME", "")/"$MIRROR_SERVER"/g' ${MIRROR_PATH}/hardware/latest/_tools/mirrors.cgi
chown -R mirror:mirror ${MIRROR_PATH}


2013/1/21 <David.Harding at brighthouse.co.uk<mailto:David.Harding at brighthouse.co.uk>>
I get this kind of error quite frequently, preceded by:
rsync: failed to connect to linux.dell.com<http://linux.dell.com/>: Connection refused (111)
I have no idea what the problem is, but I seem to get connected as often as not, so I'm not that bothered.

On the subject of the rsync script, does anyone have a reliable way of selecting just the Packages relevant to their distro(s) and particular Hardware?
I have a great long list of rsync --includes and --excludes in an attempt to limit the download to just 64-bit RHEL, the limited set of Dell HW I have, and just the latest few versions of OMSA. I figure that there must be a better way; is there?

Cheers,
dave



From:        "jean-marc vizier" <jvizier at gmail.com<mailto:jvizier at gmail.com>>
To:        "Mike Hanby" <mhanby at uab.edu<mailto:mhanby at uab.edu>>,
Cc:        <linux-poweredge at dell.com<mailto:linux-poweredge at dell.com>>
Date:        20/01/2013 16:17
Subject:        Re: [Linux-PowerEdge] Mirror OMSA repository
Sent by:        <linux-poweredge-bounces at dell.com<mailto:linux-poweredge-bounces at dell.com>>
________________________________



Hello Mike,

Thanks for the script.

It seems not to be a script problem. Mine was ok for months. For me it seems to be load problem on remote side but i'm not sure.

After some days, it's ok.

I doesn't need all omsa versions and dell platforms so i cut in all.

Best regards,



2013/1/15 Mike Hanby <mhanby at uab.edu<mailto:mhanby at uab.edu>>
Here's the script I use in case it helps, it's successfully pulling the latest 7.2:


#!/bin/bash

distrib=dell

basedir=/repo

outputdir=$basedir/mirror/${distrib}

lockfile=$basedir/${distrib}-rsync_updates.lock



if [ -f ${lockfile} ]; then

    echo "Updates via rsync already running, lock file exists: ${lockfile}"

    exit 0

fi



if [ -d ${outputdir} ] ; then

    cd ${outputdir}

    touch ${lockfile}

    # This line will mirror the entire repo, which is over 30G

    su - repoadmin -c "rsync -aHz --delete-after --delete-excluded \

      linux.dell.com::repo/ ${outputdir}/"



# 20111024 - MJH - Dell's repo doesn't contain vital symlinks to create the .htaccess files

#  Create them if they don't exist (currently in community and OMSA_ directories

   for dir in community $(ls -d hardware/OMSA_*); do

     cd ${outputdir}/${dir}

     if [ ! -L ".htaccess" ] ; then ln -s _tools/htaccess_repo_topdir .htaccess; fi

     cd _tools

     if [ ! -L ".htaccess" ] ; then ln -s htaccess_tools .htaccess; fi

     cd ${outputdir}

   done



  /bin/rm -f ${lockfile}

  # Set the permissions of the files

  chmod -R go+r,go-w ${outputdir}

else

    echo "Target directory $outputdir not present."

fi



From: linux-poweredge-bounces at dell.com<mailto:linux-poweredge-bounces at dell.com> [mailto:linux-poweredge-bounces at dell.com<mailto:linux-poweredge-bounces at dell.com>] On Behalf Of jean-marc vizier
Sent: Monday, January 14, 2013 3:09 AM
To: linux-poweredge at dell.com<mailto:linux-poweredge at dell.com>
Subject: [Linux-PowerEdge] Mirror OMSA repository



Hello,

Since 2013/01/10 i can't rsync the OMSA repository. Is there something change on the server side ?

here is the error i get :

rsync error: error in socket IO (code 10) at clientserver.c(122) [Receiver=3.0.9]

The only thing i know : on this date the OMSA 7.2 was added.

If someone has a clue ?

Regards,

Jean-Marc

_______________________________________________
Linux-PowerEdge mailing list
Linux-PowerEdge at dell.com<mailto:Linux-PowerEdge at dell.com>
https://lists.us.dell.com/mailman/listinfo/linux-poweredge

-------------------------------------------------------------------------------------------
Caversham Finance Limited, registered in England No 0785922 Registered office:
Hercules Way, Leavesden Park, Watford WD25 7GS
Caversham Trading Limited, registered in England No 05396147 Registered office:
Hercules Way, Leavesden Park, Watford WD25 7GS
BrightHouse Limited, registered in England No 06073794 Registered office:
Hercules Way, Leavesden Park, Watford WD25 7GS


-------------------------------------------------------------------------------------------
Caversham Finance Limited, registered in England No 0785922 Registered office:
Hercules Way, Leavesden Park, Watford WD25 7GS
Caversham Trading Limited, registered in England No 05396147 Registered office:
Hercules Way, Leavesden Park, Watford WD25 7GS
BrightHouse Limited, registered in England No 06073794 Registered office:
Hercules Way, Leavesden Park, Watford WD25 7GS
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.us.dell.com/pipermail/linux-poweredge/attachments/20130204/007dd694/attachment-0001.html 


More information about the Linux-PowerEdge mailing list