My home data center

datacenter000x1080

Here’s an update video of my home Data Center

Enjoy!


 

  1. Introduction
  2. Pictures & videos
  3. Detailed view of the racks
  4. Electricity
  5. Servers under vCenter
  6. Hardware and topology
  7. Home VOIP services
  8. BladeCenter
  9. Reference manuals

1 – Introduction

In another career, I taught telecommunications, operating systems and computer networks. Nowadays, I mainly teach application development for iOS in Xcode. For those of you who read French, I have an ‘Introduction to Xcode’ blog. But I still like to play with computers so, just to keep my hand in; I’ve built myself a small data center at home. So what kind of testing goes on here?

  • Hardware:  enterprise class servers – Dell, HP, IBM –  SAS, RAID 5, 6 + spare, RAID 50, 60, hot swap, UPS, F.C. SAN, Fibre switch, tape backup (IBM LTO3), …
  • OS – Linux, Windows, OpenVMS, z/OS, HP-UX, IBM AIX, …
  • Virtualization – VMware vSphere Hypervisor, vCenter server, vMotion, openStack, Xen, …
  • Network – switch, CLI, LACP, MSTP, ACL, topology, fiber optic, vlan, firewall, traffic shaper, trust and untrust LAN, IPV6, …
  • VOIP – Asterisk, VOIP, POE (Dell powerConnect 3448P), IP phones (Nortel 1230), SIP, …
  • And much more …

Enjoy your tour in my data center!

2 – Let’s look at a few pictures

Here are some pictures of the data center taken in the summer of 2014:

 datacenter04 datacenter03 datacenter01

datacenter02datacenter00

3 – Detailed view of the racks

To get a detailed view of the hardware in the data center, click on the picture below: Note: what you don’t see in this picture: IBM system P5 (AIX),  IBM AS400, HP rx2620, 2 x VAX-4000 200, Micro VAX and a  Juniper NetScreen.

Winter 2014

dataCenter-map-1920

Summer 2014

detail-datacenter-2014.07.31x1080

4 – Electricity

In no way, it would be possible to supply this kind of system without attention to electricity. My first step was to install an electrical box to provide a power of  240 volts at 40 amp, which will provide a maximum of 9.6kw/hr when needed. Note: The servers are rarely open all at once, average consumption is 1-2 kw / hr. The data center electrical box is connected to the main electrical box using an 8-3 CU wire. In addition, UPS are installed behind the rack No. 1. Electricity distribution to the servers is controlled by a Sentry power tower using a web interface.  DSC00034 DSC00040 DSC00045 Sentry power tower: srv-ptxlhf16-2-06 Sentry power tower web interface: sentry 01

5 – vCenter server

Most of my i86 servers run VMware vSphere Hypervisor.  Centralize management is obtained through vCenter server. vCenter server is running in an auto start ‘windows server 2008R2 VM’ on the HP DL380 G5.   For additional reliability, the 380 is connected to a UPS. Here’s a short video screen capture of me parsing my vCenter resources with a vSphere Client.

5.1 The map

vCenter - map

5.2 Novell server

novell server

5.3 Novell client – under windows 8.1

Capture d’écran 2014-01-30 à 20.09.37novell client win8.1

5.4 Novell iManager

Novell - Annuaire Novell - Arborescence

6 – Hardware and topology

Currently being written…

  1. Network diagram
  2. Network switches and routers (including fibre channel)
  3. Servers

6.1 – Network diagram

schema reseau maison

6.2 – Network switches and routers (including fibre channel)

– Hp 352 fibre channel switch

The 3 SAN of the data center are interconnected with a HP 352  fibre channel switch. hp-352-fibre The switch configuration is achieved trough a WEB interface: fibre channel switch And here’s a few volumes resynching in Windows server 2012 running on a PowerEdge 2850 II with 2 x 2Gb F.C. controller. RAID 5 - RESYNCHING

6.3 – Servers

Intelligent Platform Management Interface (IPMI) Most of my servers have an IPMI card (Dell DRAC, HP iLO, …) for remote monitoring and control. iLO2 on my HP.DL380-G5 (ESXi, Asterisk, …) HP-DL380-iLO-02 HP-DL380-iLO-01 DRAC5 on my PowerEdge 2950.II.09 (pfSense server): PE2950-09-DRAC Adding memory in the DL585 G5 and DL580 G5 i01 i03 i02 more to come …

Storage and NAS

Data domain DD500 – Running FreeNAS. 4 x 2TB Raid5 + 11 x 500GB Raid6 = 10.5TB of storage   dd580    

7 – Home VOIP services

What a pleasure to be able to speak, by phone, to all our friends and acquaintances, regardless of their geographical location without breaking the bank. That is why we decided to move to IP telephony. Our VOIP home network consists of 5 Avaya 1230 phones (reflash with SIP firmware), a Power Over Ethernet switch, a AsteriskNOW server – VM machine on DL380 – with the FreePBX administrative GUI and an external  SIP/ IAX2 trunk provider (voip.ms).

7.1 – Hardware

  • Access to the Internet
  • A POE Switch – Dell powerConnect 3448P

img_pro_1302_3_a

  • 5 Avaya 1230 ip Phones

Avaya-1230

  • 1 HP DL380G5 – running a VM of asteriskNow

dl380g5

7.2 – Software

Asterisk

7.3  – An  SIP/IAX2 trunk provider

Capture d’écran 2014-01-31 à 13.24.10

  • A little effort of configuration

And Voilà, it’s up and running!

8 – BladeCenter

This one is quite a beast; noisy and power hungry!  It runs on 240 volts only. There are 11 LS20 (2 * dual core AMD CPU) and 3 HS20 (2 * dual core intel CPU) blade in the bladeCenter. Most of the blades are running VMware vSphere Hypervisor. 800px-IBM_bladecenter_(front) Blades configuration is achieved trough a WEB interface: bladeCenter-Leds bladeCenter-power bladeCenter-status Configuration of  the layer 2-3 GbE integrated switch  is also achieved trough a WEB interface: bladeCenter-Switch

Reference manuals

PowerEdge-2950ii_owner’s manual_en-us MR2324.4C Management Guide Juniper Netscreen 500 Sentry PTXL v52 HP DL580 G5 HP-DL585 G5 – User manual HP Integrity RX2620 HP 9000 rp3440 user guide Power connect 3448P User’s Guide SUN sunfire t2000 admin guide SUN sunfire t2000 install guide

Advertisements

60 Responses to My home data center

  1. ilay says:

    And you have all of this in your house, wow, what country do you live in?

    • ve2cuy says:

      Yes, all this in the basement of my house.

      Québec, Canada

      Note: electricity is 0,07$ kwh.

      • Jay says:

        Is it easy to migrate to Canada? Here in Sydney, Australia, our electricity is a whopping $0.30/kWHr and is going up by 15% each year.

      • ve2cuy says:

        Jay,

        At $0.30/kWhr, electricity is quite expensive in Australia!

        The price tag of $0.07/kWhr partly explain why I’m able to install a data center at home.

        During winter time, when the temperature drops below -20c for weeks, much of the heat generated by the center, is used to heat the house.

        Alain

      • Awesome home data center! Maybe I could have something like this someday for testing and learning new things.

        Would you possibly know any IT companies around Toronto area who would be willing to take foreigners into internship? I’ve had pretty bad luck getting internships here in Finland so far and now I am on my third year on my IT engineer degree (4 year degree). Our internship periods are usually during summer season and last for about 4 months.

        Maybe little bit bold to ask like this, but any bit of help is appreciated! 🙂 I’ve also been thinking about moving to Canada at some point of my life, specially now as getting job or even paid internship is so hard here currently.

        Thanks in advance!

  2. Asim Patel says:

    i became a big fan of yours, you are superb. How can i follow you (twitter, Facebook, Mail) ?

  3. Your data center is fantastic!

  4. Frank D says:

    Great arrangement, unbelievable nerdy! Great! 🙂 Greets from Germany!

  5. I enjoyed your you tube video. I would like to see another video covering how you use all the hardware. Basically a video of you talking about everything you go over on this page. Sound lazy or me? I believe that you would talk and bring allot of insight. I’m interested in learning some of this (not Novell) and would soak in any knowledge you’d pass down. Thank you very much for what you’ve done so far. Without having access to a real data center at work I’d have no other source of information.

  6. Jul says:

    What do you use all this for?

  7. Ping : The Home Data Center: Man Cave for the Internet Age

  8. Yusuf D says:

    What i am curious is the use case for this data center? Can you elaborate what are the different apps you run?

    • ve2cuy says:

      Hi Yusuf,

      I have taught computer science for over 25 years now.

      My data center allows me to experiment with new development such as: cloud computing, virtualization, network solutions, Metal As A Service and others.

      Some servers, in the data center, serve the needs of our family : IP Phones, NAS, surveillance cameras , proxy, firewall, …

      Thank you for your interest.

      Alain

  9. J T says:

    You left out the coolest part — In the first photo, what are the two phones and lamp setting on? Looks like a MicroVAX of some kind.

  10. zaim c says:

    Do you generate money from this setup? or is it just a hobby?

  11. Dufflepod says:

    Beautiful. Just beautiful.

  12. Zaman Mecci says:

    Hi Alain,

    I run a technology media startup company techstorey.com which provides insights in the field of
    technology via interviews from experts. I will appreciate if you give me an opportunity to interview you and publish about your project. Upon your agreement, I will study your project in detail and I might come across some questions which are technical in nature. If it is ok I might also publish your network diagram. If not, I can create my own.

    I will be glad to provide any further information which you might need. I look forward to hearing from you.

  13. andrew says:

    Amazing site, I favourited as I enjoyed it. Do you run on 3-phase power in Canada, domestic power in the UK is only single phase, although I’m sure I remember reading somewhere that Canada is 3-phase 600v?

    • ve2cuy says:

      Hi Andrew,

      3 phase 347/600 V. is available for industrial needs.

      At home, we use 1 phase 120 V/240 V.

      Some of my servers are connected to 240 volts but most of them are on 120 Volts.

      Thank you for passing by and take care!

      Alain

      • andrew says:

        Amazing, it’s great to read up on your site. I make small amounts of money off my website and youtube video which help covers the fees I incur. Luckily they don’t cost much but I want to develop my IT skills are look more into home datacentres too.

  14. ky says:

    How can i have 60tb of storage with equipment hooked straight to my laptop what is it is it san or das or how do i do this and access it through my laptop?

  15. l1byan says:

    can I live there?

  16. Ping : The Home Data Center Project - TechStorey

  17. Zaman says:

    Alain,

    What is your total electricity bill?

  18. Justin says:

    haha thats so funny. You should get out more. No but seriously, Do you actually use that equipment for anything? Seems more of a work of art than useful. Very impressive 😉

    • ve2cuy says:

      Hi Justin,

      Yes, I should get out more!

      The fact remains that, in Quebec, the average temperature, during winter, is -20c .

      So, we find things to do inside.

      Thank you for the ’work of art’ reference!

      Actually, the datacenter is used for something useful:

      Left rack:

      DELL PowerEdge 2950 Raid 6 – 8 x 2 TB – Home NAS (movies, music, photos)

      DELL PowerVault 124t – NAS backup

      8 x PowerEdge – vmware (vcenter) experimentation and live migration.

      Middle rack

      HP 380 GEN7 – Asterisk (home IP phones), pfsense – proxy and protection –

      8 x HP 380 – MAAS – openstack – experimentation.

      Right rack

      Itanium and Alpha Proc – OpenVMS ware.

      Being an IT teacher, I use the material of my datacenter to test advanced technologies.

      Take care!

      Alain

  19. tanveer says:

    Your home data center is amazing can you tell me from where can I get all these equipments, hardware plus software to build my own. With starting from five servers what will be the total cost for it.

  20. Fernando says:

    amazing Data Center, thanks for share belive me useful info understanding all of this!

  21. Optional says:

    Amazing home setup! 🙂
    What about the internet connection?

  22. joe says:

    Interesting that I found this while waiting for my friend that’s an electrician that’s coming over to install a 240v breaker in my breaker box. I picked up a SGI Onyx 2 that I restored and am adding it to my computer collection. Currently running a SGI Origin server and an Octane 2 along with a plethora of networked Mac and Pc’s. I’ve been in graphics and animation since the early 90’s and always wanted SGI equipment. Someone was giving away the Onyx so I took it to keep it out of a landfill.

    Nice setup you have there!

  23. ashish says:

    Hello ve2cuy,
    I am from India and I have this long time dream of setting up a internet source for my plan to start a mini private data center to power a gaming portal, internet/browsing center as internet connectivity is very very limited in my area. I just want to say that you have an amazing set up within your home and I felt that a set up like this would be a great start my idea. I would accept any basic suggestions for my idea and your recommendations.
    Thank you!!

  24. R9CBM says:

    Just wonderful! Is there static electricity and dust?

    • ve2cuy says:

      Hello R9CBM,

      Very little dust thank’s to a heat exchanger and a HEPA filter.

      Ye,s there is static electricity, mostly during winter – So, I have to stay vigilant.

      Alain

  25. Hi,

    I have been following you past 1 week.I have a 1 HP Proliant ML110 g7 server, 5TB HDD, 16GB RAM. I need to build a small home Datacenter for my 10 e-commerce website. I ll need a backup server too. It should be very fast and should be hacker-proof. I can spend more but it should be in my budget.

    I have some questions.

    a) What are hardware specification I need to follow ?
    b) How much bandwidth I will require ?
    c) How much electricity it will require ?
    d) Which plan I should choose from our local ISP ?
    e)

    Any comment will be appreciated.

    Thanks.

  26. Sandeep Saxena says:

    Simply Awesome. I wish I could make something similar for myself. But I realize that first I need a lot of learning before I invest. I do have a very small setup consisting of few desktops, laptops, routers and a Netgear NAS device but would love to scale this to something which you have. A great inspiration for me. Would love to listen and see more from you.

  27. Happy says:

    Hi ve2cuy ,
    Please i love your networking skills. I want to switch to networking please…

  28. archiepancakes says:

    jesus fucking christ! this really is pretty cool but whats your reason for having/managing this monstrosity? how much time/money are you spending on this? why not just use amazon/google for hosting?

  29. Hallis says:

    Hello, how are you

    I like your data center and it inspired me to ride one here in Brazil to offer hosting service

    You have an email to nois two talk and take some doubts ?

    Congratulations for your data center very beautiful and functional !!!

  30. Anand says:

    May i know for what purpose you are using this data center. and can you please list out total price taken for this whole data center setup.

  31. Panagiotis G says:

    Hi, Great network. I got a question. I wasn’t able to figure out the complete list of internal servers and switches. Ok you connect an HP Fiber switch but on what speed? Also, I can see that you are using some Cisco Devices , could you please let me know the model numbers and for what purpose you are using them? Also, one general question, what is the purpose of all this system? Development? playing arround with Labs or commercial infrastructure?

    Thank you

  32. Ping : From Electricity to Fibre: Powering Up Your DIY Home Data Centre |

  33. Ping : From Electricity to Fibre: Powering Up Your DIY Home Data Centre – Earn 24 7 365

Laisser un commentaire

Entrer les renseignements ci-dessous ou cliquer sur une icône pour ouvrir une session :

Logo WordPress.com

Vous commentez à l’aide de votre compte WordPress.com. Déconnexion / Changer )

Image Twitter

Vous commentez à l’aide de votre compte Twitter. Déconnexion / Changer )

Photo Facebook

Vous commentez à l’aide de votre compte Facebook. Déconnexion / Changer )

Photo Google+

Vous commentez à l’aide de votre compte Google+. Déconnexion / Changer )

Connexion à %s

%d blogueurs aiment ce contenu :