Charla 23-11 Ummet'2000

Charla del 23/11/2000
Log de la conferencia. Se han suprimido las líneas correspondientes a entradas y salidas de diferentes personas en el canal durante la conferencia

[22:11] (Fernando) Hola,
[22:11] (Fernando) estamos muy contentos de poder presentarles hoy aquí a David Santo Orcero.
[22:11] (Fernando) Nació en Málaga, pero actualmente trabaja en Brasil.
[22:11] (Fernando) Esta haciendo su doctorado en biofísica molecular estructural en IBILCE/UNESP.
[22:11] (Fernando) Está integrado en un proyecto de investigación sobre métodos
[22:11] (Fernando) computacionales para la biofísica molecular; tiene que ver con el cálculo
[22:11] (Fernando) de estructuras tridimiensionales de proteínas.
[22:11] (pask) me parece que le conozco ..
[22:11] (Fernando) Como parte del resultado de sus
[22:11] (Fernando) investigaciones se desarrollará una aplicación para la realización de estos
[22:11] (Fernando) cálculos, que se planea distribuir como software libre.
[22:11] (Fernando) Además de su investigación, creó y mantiene un cluster Mosix que se usa
[22:11] (Fernando) para el estudio de las toxinas de las serpientes.
[22:11] (Fernando) También hemos podido leer sus artículos en algunas publicaciones linuxeras.
[22:12] (Fernando) Sólo me queda agradecer a David y a todos ustedes su presencia aquí.
[22:12] (Fernando) Su conferencia se titula State of art and perspectives of high-performance
[22:12] (Fernando) Linux clustering
[22:12] (Fernando) David ...
[22:12] (irbis) Hola, Fernando
[22:12] (irbis) Echamos a andar?
[22:12] (Fernando) adelante
[22:12] (irbis) Buenos dias.
[22:13] (Neo) que tal
[22:13] (irbis) Todo el mundo habla espanhol? Alguiem portugues? Nihongo hanamasu ka?
[22:13] (irbis) La conferencia sera en ingles,
[22:13] (jahoitl) listo
[22:13] (jahoitl) Por que
[22:13] (viper) por?
[22:13] (Neo) ok
[22:14] (irbis) pero despues respondere a las preguntas en el idioma que cada uno quiera.
[22:14] (Martha) hablamos español
[22:14] (irbis) Un segundo, que eche a andar el script (expect+dsirc)
[22:14] (ciph3r) te has preparado ya la charla en ingles???
[22:15] (irbis) Comenzare con algunos conceptos basicos de sw libre, por si alguien no los conoce, y despues entramos en el meojo.
[22:15] (MJesus) plas plas plas plas plas plas plas plas plas plas
[22:15] (Fernando) plas plas plas plas plas plas plas
[22:15] (Fernando) plas plas plas plas plas plas plas
[22:15] (Fernando) plas plas plas plas plas plas plas
[22:16] (MJesus) plas plas plas plas plas plas plas plas plas plas
[22:16] (MJesus) clap clap clap clap clap clap clap clap clap clap
[22:16] (MJesus) plas plas plas plas plas plas plas plas plas plas
[22:16] (MJesus) clap clap clap clap clap clap clap clap clap clap
[22:16] (jahoitl) Ok, is ready
[22:17] (MJesus) los duenfdes en el script
[22:17] (irbis) Llevo toda la tarde probandolo, y ahora no loga en el servidor irc.. un segundo.
[22:17] (MJesus) ya paso el cuarto de hora de cortesia... asi que el que llegue tarde es por su cuenta
[22:18] (Neo) ok no problem
[22:20] (Fernando) bueno
[22:20] (Fernando) mientras arranca el escri
[22:20] (Fernando) recordarles que mañana tendremos otra interesante conferencia
[22:20] (Fernando) sobre seguridad
[22:20] (Fernando) a cargo de Borja Marcos de la Puerta
[22:20] (Fernando) y que se titula
[22:20] (MJesus) plas plas plas plas plas plas plas plas plas plas
[22:20] (MJesus) plas plas plas plas plas plas plas plas plas plas
[22:20] (MJesus) plas plas plas plas plas plas plas plas plas plas
[22:21] (Fernando) Los enanitos del bosque han desconfigurado mi sistema
[22:21] (Fernando) recordarles tambien
[22:21] (Fernando) que a través de
[22:21] (Fernando) umeet.uninet.edu/spanish/des.html
[22:21] (Fernando) pueden acceder a las conferencias celebradas hasta el momento
[22:22] (Fernando) que han sido grabadas y puestas a su disposición en la web
[22:24] (irbis) ``Es de bien nacidos ser agradecidos'' - Spanish proberb
[22:25] (irbis) There are some persons and entities that I must to cite here. Maybe you will be here on the next version...
[22:25] (irbis) My wife Maria Francisca Ribeiro de Araujo Santo Orcero, for her understanding and support.
[22:25] (irbis) My father Antonio and my mother Maria del Carmen, for all.
[22:25] (irbis) FAPESP, for giving to me a grant for my Ph.Dr., on the project ``Computational methods for molecular biophysics''. Using the reserva
[22:25] (irbis) técnica of this grant -arround R$4000- I built the first Mosix cluster.
[22:25] (irbis) Linux kernel wizards, FSF, LATEXteam, KDE team and KlyX team, for so much wonderful software that I use daily.
[22:25] (irbis) Is this talk useful for me?
[22:34] (Irbis_) In computing science, the most of the problems can be organizated on families, with similar solution. This talk is useful for you if you:
[22:34] (Irbis_) Develop basic science using computers for this.
[22:34] (Irbis_) You are involved on ``number-crushing'' or ``big-challenge'' problems, you need more power and you have no more money.
[22:34] (Irbis_) You find interesting to gain knowledge on an exiting area, that is going to be one of the more important engines of the development of
[22:34] (Irbis_) humankind on XXI century.
[22:35] (Irbis_) Overview of the talk
[22:35] (Irbis_) A overview of we are going to talk on the first half is:
[22:35] (Irbis_) Supercomputers facilities for cheap cost. We are going to compare some Linux clustering techniques, their strong and weak points, and which
[22:35] (Irbis_) one can be best for a particular problem.
[22:35] (Irbis_) Some about what I really do. Maybe the speaker is doing something interesting. ;-)
[22:36] (Irbis_) If we have time -I do not thing so- some neat hints about how to administrate clusters for high-performance or high-availability where you
[22:36] (Irbis_) are the only one computer science guy. Dilbert rulez. ;-)
[22:36] (Irbis_) Finally, If I do not talk something, it would be because short of time or short of knowledge. Do not doubt to share knowledge on the Q&A session at
[22:36] (Irbis_) the end of my presentation. Sharing knowledge is the basis of science and free software movement.
[22:36] (Irbis_) On the second half we are going to do a live demostration of the instalation and configuration of a Mosix cluster.
[22:37] (Irbis_) Some basic concepts
[22:37] (Irbis_) The free software paradigm, GNU and FSF
[22:37] (Irbis_) After some troubles with his institute's software development policy, MIT hacker Richard Stallman founded FSF -Free Software
[22:37] (Irbis_) Foundation-.
[22:37] (Irbis_) He created the GPL licence and the copyleft. The sense of our ``free'' is about freedom of speech, not Free beer.
[22:37] (Irbis_) Free software movement claims that development of software must be free. You must be allowed to improve the code of the others, as you
[22:38] (Irbis_) keep the register of their work.
[22:38] (Irbis_) Free software movement looks as basic science development paradigm. No one can patent Newton's Law. You discover software -what is
[22:38] (Irbis_) about knowledge-, not make software -what is about industrial machinery-.
[22:38] (Irbis_) Free Software way of thinking is common on Computational Chemistry, only look at CCP collaborative projects of UK government, CCL,
[22:38] (Irbis_) QCPE -born on 50's, before Stallman's FSF-, GAMESS, MOPAC...
[22:38] (Irbis_) You use lots of FSF tools; GNU C compiler, Emacs...
[22:38] (Irbis_) The Bazaar Model
[22:39] (Irbis_) It is the theoretical basis that try to explain the fast growing of some pieces of Free Software -Apache Web Server, Linux Kernel-.
[22:39] (Irbis_) Proposed by Eric Raymond, an open source evangelist.
[22:39] (Irbis_) Development of traditional Software is like building a Cathedral. Its management is centralized, as the masons only build whatever they were
[22:39] (Irbis_) said to be built.
[22:39] (Irbis_) The new model: Bazaar model. Decentralized management, lots of people doing its own.
[22:39] (Irbis_) It has historical cases on Computational Chemistry: GAMESS, CCP4. And It worked!
[22:39] (Irbis_) Bazaar development has one problem: nobody does what nobody want to do -eg, there are lots of GUIs; but FSF GPLed kernel, Hurd, ten
[22:39] (Irbis_) years after the work begun, is far to be completed-.
[22:39] (Irbis_) Eric Raymond's model has flaws: it does not explain Motzilla's disaster.
[22:39] (Irbis_) One of its stronger points is the mass of parallel beta-testers with knowledge and ability of correct and improve the source code.
[22:39] (Irbis_) In the begining, there was free software...
[22:40] (Irbis_) The free software, and the concept of sharing it is not strange to science.
[22:40] (Irbis_) Sharing it is not a strange concept on science. Sharing knowlede is the backbone of the advance of science and quality of life of humankind.
[22:40] (Irbis_) Do you imagine a patent over the weel?
[22:40] (Irbis_) Do you imagine a patent over the writting?
[22:40] (Irbis_) Do you imagine a patent over the potery?
[22:40] (Irbis_) Do you imagine a patent over the gravitation law?
[22:40] (Irbis_) Patents, and any limitations to the expansion of knowledge, used to be strange to basic science.
[22:41] (Irbis_) And software was always considered by scientists as a piece of knowledge.
[22:41] (Irbis_) The fever of Linux's fashion, and some Linux Miths.
[22:41] (Irbis_) One day, by a mistery reason, Linux was no more a extravaganza and became cool. But we must renember that:
[22:41] (Irbis_) Linux is a flavour of Unix: With its strongness and its weakness.
[22:41] (Irbis_) Managing Linux needs deep knowledge: As you have more power on your hands, you must to learn how to manage it. The good tools of
[22:41] (Irbis_) administration, as Linuxconf, make lots easier to manage Linux networks for the expert, but, for example, a person that knows nothing about
[22:41] (Irbis_) DNS services, can't configure a DNS server for a domain.
[22:41] (Irbis_) Using Linux needs some knowledge: No way. As in the case avove, as you have more power, you must know what you are doing.
[22:42] (Irbis_) All this affirmations are not ``cool'' for non-technical users. And most of them are mixed with Linux miths. Anyway, nobody makes
[22:42] (Irbis_) MS-Windows better than M$, thus if we begin to modify all the things that make Linux powerfull, changing for Windoze things, we will have more
[22:42] (Irbis_) Windows again.
[22:42] (Irbis_) Personally, I have heared for a important and famous person of Linux orbit "X-Window sucks". He also have said "Linux sucks" and other things
[22:42] (Irbis_) like this, thus it looks that all that it is not his product sucks. Anyway, it is a strong stream of opinion on some circles that Linux should look and
[22:42] (Irbis_) act like Windows, but being free. It is the ``grandma is at the keyboard'' stream.
[22:42] (Irbis_) By other way, the power and the flexibility of X-Protocol and Unix capacities will save the day on the day-to-day use on a laboratory. We change
[22:42] (Irbis_) some time of learning for better features.
[22:43] (Irbis_) On the other side, important distributions, with M$-Like instalation procedures, needs more time and knowledge to be instaled than other more
[22:43] (Irbis_) user friendly -IMHO, only more pretty- distributions.
[22:43] (Irbis_) Many times you are forced to install one operating system or distribution acording to your boss's opinion about what is more user friendly. No way
[22:43] (Irbis_) that most of them looks the same after installing. It doesn't matter.
[22:43] (Irbis_) Maybe, with all this fever of Linux's fashion, people that really loves Linux and Free Software continues being misfits. But nowadays only these
[22:43] (Irbis_) misfits can take the best of the hardware.
[22:43] (Irbis_) The right to read
[22:43] (Irbis_) When our freedom was in danger, Stallman did an excelent article called ``the right to read''. This article, far from loosing actuality, each day is
[22:44] (Irbis_) more important, because we are arriving to the situation descrived on it. When I read that italian society of authors wanted to cut the free libraries,
[22:44] (Irbis_) I renembered Stallman's article and began to feel afraid.
[22:44] (Irbis_) The 4 freedoms
[22:44] (Irbis_) The objetive of Stallman since then was to create a free operating system, based on his believes on Freedom. He said that all software must to have
[22:44] (Irbis_) the follow freedoms:
[22:44] (Irbis_) Freedom 0: The freedom to run the program, for any purpose. (It is somewhat stupid, but nowadays it must being specificated because this
[22:44] (Irbis_) freedom is in trouble, as all the others)
[22:44] (Irbis_) Freedom 1: The freedom to study how the program works, and adapt it to your needs. Access to the source code is a precondition for this.
[22:45] (Irbis_) Freedom 2: The freedom to redistribute copies so you can help your neighbour.
[22:45] (Irbis_) Freedom 3: The freedom to improve the program, and release your improvements to the public, so that the whole community benefits.
[22:45] (Irbis_) Access to the source code is a precondition for this.
[22:45] (irbis) La publicidad del partido termina, comenzamos con lo bueno. ;-)
[22:45] (Irbis_) Linux as a supercomputing tool
[22:45] (Irbis_) Free operating systems that can be used as supercomputing tool
[22:45] (Irbis_) There are many free Unix flavours. They have diferent free licences; but the more common licences are BSD and GPL, completly free. There are
[22:45] (Irbis_) slight differences between them on the performance; but they are very similar for a scientist. The more importante diference is on the instalation
[22:46] (Irbis_) process, because Linux has the easiest instalation process. The most common free operating systems that can be used for cheap supercomputing
[22:46] (Irbis_) are:
[22:46] (Irbis_)
[22:46] (Irbis_) Linux: Under GPL licence, it is the most common free operating system available, and the easier one to install. It has lots of different
[22:46] (Irbis_) distributions, each one with its strongness and weakness. It is the best choose for a desk scientific workstation on a cheap computer. More
[22:46] (Irbis_) information can be found at http://www.linux.org/.
[22:46] (Irbis_) Hurd: It is the same system than Linux, but it uses the Hurd kernel, developed by FSF; Linux uses the kernel developed by Linus Torwalds.
[22:46] (Irbis_) The Hurd is built on top of CMU's Mach 3.0 kernel and uses Mach's virtual memory management and message-passing facilities. It has a
[22:46] (Irbis_) really creative design, so creative that at this time it does not work well. Anyway, It may be an alternative on the future. Linux binaries are not
[22:47] (Irbis_) compatible with Hurd ones, but most of them can be recompiled for Hurd. Optimal for research on Computer Science, other uses are
[22:47] (Irbis_) discouraged at this moment. It is GPLed. It can be found at http://www.gnu.org.
[22:47] (Irbis_) FreeBSD: Free flavour of Unix, under BSD licence. It has a faster implementation of TCP/IP stack -with a great diference-. Anyway, it is
[22:47] (Irbis_) harder to install. Nearly all the aplications for Linux can be recompiled for FreeBSD. Maybe is the best choose for servers -like SAMBA,
[22:47] (Irbis_) NFS or Web servers-. It can be found at http://www.freebsd.org/.
[22:47] (Irbis_) NetBSD: Free flavour of Unix, under BSD licence. Really mature and stable. It is the free flavour of Unix more portable, thus more ported to
[22:47] (Irbis_) more platforms. Currently, it has versions on alpha amiga, arc, arm26, arm32, atari, bebox, cobalt, hp300, hpcmips, i386, luna68k, mac68k,
[22:47] (Irbis_) macppc, mipsco, mvme68k, news68k, newsmips, next68k, ofppc, pc532, pmax, prep, sgimips, sh3, sparc, sparc64, sun3, vax, x68k. The best if
[22:47] (Irbis_) we want easily port Unix to a strange machine. It can be found at
[22:48] (Irbis_) http://www.netbsd.org/
[22:48] (Irbis_) OpenBSD: Free flavour of Unix, under BSD licence. It is, by distance, the most secure operating system that can be found. It is the only
[22:48] (Irbis_) operating system which source code has being fully audited. It had three years without a remote hole in the default install, and only one
[22:48] (Irbis_) localhost hole in two years in the default install. It supports binary emulation of most programs from SVR4 (Solaris), FreeBSD, Linux,
[22:48] (Irbis_) BSD/OS, SunOS and HP-UX. It is the best for machines that can be attacked, as mail servers, DNS servers, NIS servers, firewalls... More
[22:48] (Irbis_) information at http://www.openbsd.org/.
[22:48] (Irbis_) Strong points of Linux as supercomputing tool
[22:48] (Irbis_) Nowadays, Linux is the best operating system to do supercomputing, high-availability uses or high-performance uses with low budget.
[22:49] (Irbis_) Existence of free Fortran 77 compilers. The most of the scientific libraries are in Fortran, and the most of the scientific programs are also
[22:49] (Irbis_) in Fortran, due to historical reasons. Most of the computational chemistry programmes are written in Fortran, and nobody at this moment is
[22:49] (Irbis_) thinking about porting the gigantic QCPE dinosaurs-packages to C.
[22:49] (Irbis_) Existence of free C compilers. C is fast and cool. And today there are lots of people developing code using C.
[22:49] (Irbis_) Stability. A real calculation on computational chemistry problem need lots of time to be completed. Some problems, like folding problems,
[22:49] (Irbis_) are especially CPU-intensive. If the machine hangs all day, you can not use the machine for serious research. Linux is really stable, and that
[22:49] (Irbis_) thing allows you to do long simulations on your PC desktop.
[22:49] (Irbis_) Performance. Many workstations are known for the low user time response. A cheap PC may need 40 minutes of CPU, but you wait only 45
[22:49] (Irbis_) minutes. A AIX workstation used by M.D. students, Ph.D. students, postdocs, researchers, will need 15 minutes of CPU, but you will wait a
[22:49] (Irbis_) couple of hours. Final performance is better.
[22:49] (Irbis_) Availability. All of us have a PC on our desktop. Few of us has an parallel supercomputer on our desktop. Anyway, if you want the
[22:49] (Irbis_) supercomputer and you have the money, you should wait long customs delays and taxes. You can buyjust-in-time a PC on any store, and
[22:50] (Irbis_) maybe your work will delay more, but it will be finished before the supercomputer is here.
[22:50] (Irbis_) Availability of PVM. There exists a complete and efficient implementation of PVM library. PVM is free and well-documented, and it is
[22:50] (Irbis_) available as Debian, RPM and tallball, with lots of free documentation and its wrappers to Fortran. Some Linux distributions include PVM
[22:50] (Irbis_) -SuSE, RedHat PowerTools, Extreme Linux project-. Usually it is easier to recompile a parallel package for a pile of cheap Pcs with Linux
[22:50] (Irbis_) without native support and documentation, than recompiling the same package for a supercomputer.
[22:50] (Irbis_) Availability of MPI. As on the case of PVM, there exists lots of applications that make use of MPI and that can be directly recompiled under
[22:50] (Irbis_) Linux. MPI's situation is completely different than PVM's one. On PVM we had only one full implementation. On MPI, we have some
[22:50] (Irbis_) different implementations. All of them use to work fine under simple applications; but some applications can need features for which a
[22:50] (Irbis_) determined implementation lacks of support. The four most common free implementations of MPI standards are MPITCH, LAM, CHIMP
[22:50] (Irbis_) and UNIFY. Unify is somewhat special. It is a PVM wrapper for MPI calls, that allows as emulate a MPI as mix PVM and MPI calls on the
[22:50] (Irbis_) same code.
[22:50] (Irbis_) Supercomputing facilities. Finally, Linux have some supercomputing facilities. The most important but less appreciated is that Linux
[22:51] (Irbis_) support most of Intel-based multiprocessor motherboards as part of the stable line of Kernels. But this facility is not the only one. We have
[22:51] (Irbis_) two cheap-supercomputer projects, Beowulf project -from CESDIS, NASA- and Mosix project -from the Hebrew University of Jerusalem-.
[22:51] (Irbis_) The two ones use completely different points of view.
[22:51] (Irbis_) Optimal MIPS/$. The most important one. Ask for the price of a Linux box. Ask for the price of a supercomputer. Look for the cost for
[22:51] (Irbis_) MIP. Only as a example, we can see the table that come with this article. Other example: The last 9 of March of 1999, on LinuxExpo 1999,
[22:51] (Irbis_) Beowulf class supercomputers outperformed to Cray supercomputer. The mundial record of PovRay -an important ray tracing application,
[22:51] (Irbis_) that involves heavy floating point calculations- benchmark, hold by a Cray processing T3t-900-AC64 of $5.5 million, was broken by a
[22:51] (Irbis_) Beowulf class supercomputer of $150,000. The Beowulf cluster rendered the image three times faster that the Cray supercomputer. There are
[22:51] (Irbis_) hundreds of other examples.
[22:51] (Irbis_) There exists other advantages, but form part of the advantages of Linux and Free software. Price of hardware, spare parts and reparations,
[22:51] (Irbis_) on-line support, global quality... Sure you know lots of them.
[22:52] (Irbis_) Problems with Linux as a supercomputing tool
[22:52] (Irbis_) Anyway, there exists limitations of Linux as a supercomputing tool. The most important ones are:
[22:52] (Irbis_) Lack of any free Fortran parallelizing compiler.
[22:52] (Irbis_) There does not exist a free Fortran-90 compiler. Fortran 90 is not as popular as its antecesor, but it is broadly used on scientific media.
[22:52] (Irbis_) Weakness of MPI. At this time, there exists some problems with free implementations of MPI, that lacks of support of threaded MPI
[22:52] (Irbis_) extensions. This is a serious problem, because several important computational chemistry parallel packages uses threaded MPI as programme
[22:52] (Irbis_) API.
[22:52] (Irbis_) Weak knowledge about what means Linux and free software. It is true that Linux has been strongly supported by community, but at this
[22:52] (Irbis_) time there are lots of people that do not want hear about Linux, and consider Linux ``something bizarre of the computer guy''.
[22:53] (Irbis_) Number-crushing and supercomputing with Linux
[22:53] (Irbis_) Linux is used usually on machines of low power. This does not means that Linux can be used in an Alpha processor, but that it is not the more
[22:53] (Irbis_) frecuent scenario. The most frecuent scenario is having no enought budget.
[22:53] (Irbis_) Using Alpha processors have also some other problems. Some countries, as Brazil, have a draconian politics of importation. Importation and
[22:53] (Irbis_) customs' steps may delay obtain the machine months, sometimes nearly a year.
[22:53] (Irbis_) Customs taxes also uses to be extremily high, and somewhat absurd. As an example, I was taxed 5 months ago for 400US$ for bringing to Brazil
[22:53] (Irbis_) my 4-years-old pentium!. The worst is that the same brazilian law assures that I hadn't to pay nothing, but you are phoned and they say to you
[22:53] (Irbis_) that you will lost your computer if you don't pay in a week, and you finish paying to recover your HD and its information.
[22:54] (Irbis_) There are some other dark histories on customs, like trying to get a camera for forgeiner tourists -particulary, my father's video camera-, and so
[22:54] (Irbis_) on...
[22:54] (Irbis_) Bring the hardware using a foundation avoid you to pay taxes, but you need nearly a year between you pay and you get the machine.
[22:54] (Irbis_) Then, sometimes it is not only prize. It is availability. You can not be one year stopped, because finacial organizations that granted to you don't
[22:54] (Irbis_) what to know why you spend a year without publishing.
[22:54] (Irbis_) There are other countries, like Cuba or China, where some supercomputer facilities have limitations for exportation from the US.
[22:54] (Irbis_) Linux, as it is available on high-availability machines, it is perfect for that kind of countries. Neither the most close goverment can block science
[22:55] (Irbis_) with importation politics, neither any country can block for you to buy a machine to do science because you are on his list of ``non-fair-with-me''
[22:55] (Irbis_) countries; because you can always go to the computer shop of the next door, buy a pile of PCs and begin to do supercomputers, without need neither
[22:55] (Irbis_) greats amounths of money nor ask permision to the local goverment or to US goverment.
[22:55] (Irbis_) And the best: you don't need to pay the "M$ tax".
[22:55] (Irbis_) A Linux box has a reasonably use of the calculus power of a single machine. M$ uses to say that it is not true, but it is best to try NT and Linux on
[22:55] (Irbis_) the same machine that earing M$ FUD techniques.
[22:55] (Irbis_) By other way, you can't reach the performance of an Alpha with one single pentium. Sorry, but it is the reality. Linux is not Lourdes neither Fatima
[22:55] (Irbis_) sanctuary. Let's see how can we get more power.
[22:56] (Irbis_) SMP support
[22:56] (Irbis_) SMP support of Linux have a very bad press. I really do not know why, because I work dayly with that kind of machines, and they ARE stable if the
[22:56] (Irbis_) memory is good. And if the memory is bad they are unestable with all OS, because it is a hardware problem.
[22:56] (Irbis_) It is true that Linux do not support the must out of the line hardware SMP facilities. It is also true that it has support only for NT (neither 95, nor
[22:56] (Irbis_) 98).It is also true that I had never known anybody with such hardware. If I could spend so much money, I would buy an Alpha processor, and
[22:56] (Irbis_) stop playing with Intel toys.
[22:56] (Irbis_) It is also true that SMP support is Beta. But "Beta" is a relativistic term. Masquerading has been beta code for years, and I have been used dayly
[22:57] (Irbis_) masquerading for years without any trouble or bug. Other operating sistems have release code for its internal functions, and hangs twice per day, on
[22:57] (Irbis_) the best of the situations, in a production environment.
[22:57] (Irbis_) SMP is a simple way to do parallelism on Linux. But it is far away to be the cheaper. Let's see others.
[22:57] (Irbis_) Clustering
[22:57] (Irbis_) Really, it is difficult to find one kind of aplication that could not take advantage of using clustering tecnologies. From high-end computer -that
[22:57] (Irbis_) needs clustering for performance- to the low end user -that would be beneficied on data recovery- all users can be beneficiated of a inteligent
[22:57] (Irbis_) cluster aplication.
[22:57] (Irbis_) Let's see what kind of clusters we can find, and for what is useful each one.
[22:58] (Irbis_) Beowulf, doing supercomputing with a pile of PCs
[22:58] (Irbis_) Beowuf is an architectural model of cheap supercomputing, based on GNU/Linux and other free software comodities. It was proposed by
[22:58] (Irbis_) researchers of the Center of Excellence in Space Data. Originaly, it only needs common hardware, but it works better with non-ethernet network
[22:58] (Irbis_) solutions, because with an ethernet network cann't scale more that 16 processors.
[22:58] (Irbis_) Performance's analysis of Beowulf class supercomputers
[22:58] (Irbis_) Comparative of NAMD performance
[22:58] (Irbis_) (kool table)
[22:59] (Irbis_) Execution time in seconds for a system of 36573 atoms, with a cut-off of 12A.
[22:59] (Irbis_) Conclusions:
[22:59] (Irbis_) Scalability of results. Fairly good, but could be best using a Gb net, better than Fast Ethernet.
[22:59] (Irbis_) Comparation between the result of PII table and above table. It looks that you need the double of processors to reach the same
[22:59] (Irbis_) performance, than a O2000, and the same processors to reach a T3E.
[22:59] (Irbis_) SGI O2000/Linux ratio; Linux Cluster with the double of processors.
[22:59] (Irbis_) Time: for 1 procs in O2000: 1.28; for 2 procs in O2000: 1.28; for 16 procs in O2000: 0.87.
[23:00] (Irbis_) Excellent! Beat O2000 with se same prize!
[23:00] (Irbis_) Cray T3E/Linux ratio; Linux Cluster with the same number of processors
[23:00] (Irbis_) Time: for 2 procs in T3E : 0.95; for 4 procs in T3E : 0.94; for 8 procs in T3E : 0.94; for 16 procs in T3E : 0.8; for 32 procs in T3E : 0.68
[23:00] (Irbis_) SIMPLY MARVELOUS!!!
[23:00] (andres) d
[23:00] (Irbis_) The great flaw of the Beowulf arquitecture
[23:00] (Irbis_) The Beowulf architecture, in the case of an optimal load balancing between the different process of the different nodes, assures the optimal
[23:01] (Irbis_) performance of the application at low cost. But, unfortunately, this case is not always the best possible stage. The most common stage is that some
[23:01] (Irbis_) nodes has CPU-bounded jobs, some nodes has IO-bounded nodes, and that is in that way because the born and the finish of process. That means
[23:01] (Irbis_) that some nodes can be subused meanwhile other nodes might be overloaded.
[23:01] (Irbis_) that some nodes can be subused meanwhile other nodes might be overloaded.
[23:01] (Irbis_) The scenario can be worse. In the case of nodes of different performance, the most powerful nodes uses to be misused on normal task partitions.
[23:01] (Irbis_) And in the case of multiuser environments, the problem is that we have no control about the load of the different nodes.
[23:01] (Irbis_) The best would be having the best of Beowulf clusters, but with load balancing. That is where Mosix clusters plays a great role.
[23:01] (Irbis_) The Mosix architecture
[23:01] (Irbis_) The Mosix architecture was developed by prof. Ammon Barak team on Hebrew University of Jerusalem, and it gives to Linux clusters the better of
[23:01] (Irbis_) clustering. The features of Mosix are:
[23:02] (Irbis_) Automatic load balancing of process. Mosix asures the best use of all the resources of the cluster;
[23:02] (Irbis_) Memory ushering. Process will run wherever they have space, for not to swap;
[23:02] (Irbis_) Descentralized control There is not a central authority;
[23:02] (Irbis_) Common interprocess space. A Mosix cluster looks like an alone machine to the user;
[23:02] (Irbis_) Backwards compatibility. It can be used PVM, MPI and sockets; all the clustered and Beowulf applications runs on a Mosix without
[23:02] (Irbis_) modifying code;
[23:02] (Irbis_) Fork and forget programming model. Creating a Mosix parallel aplication is so easy as calling the fork() Unix POSIX function;
[23:02] (Irbis_) Transparent to the user; and, finnaly, the applications can give clues of where is their bottlenecks, for Mosix optimize them; but this is not
[23:02] (Irbis_) compulsatory.
[23:02] (Irbis_) Performance of the Mosix architecture
[23:03] (Irbis_) Two comparations -figures at the end of the article-; PVM vs. Mosix, a test battery of different applications with different runtimes, that proves
[23:03] (Irbis_) that a Mosix outperforms PVM on a cluster; and running a molecular dynamics simulation of 1000000 atoms, on a Pentium Pro cluster with
[23:03] (Irbis_) Ethernet, that proves that Mosix outperforms a SP/2. For 8 node, Mosix is 4 times faster than SP/2. When we use 16 nodes, we find again a
[23:03] (Irbis_) bottleneck on the network layer, that can be easly broken using Fast Ethernet -A 30R$ communications card!!-. Mosix is a fully-competitive
[23:03] (Irbis_) platform for CPU-bonded parallel tasks.
[23:03] (Irbis_) Network interfaces for Linux clusters
[23:03] (Irbis_) When we are going to build a dedicated cluster, it is not a good idea to use a common Ethernet adaptor. Neverthless it is cheap and reliable, big
[23:03] (Irbis_) bandwidth applications will find a bottleneck here when we have 16 nodes or more. There are some good network interfaces, more expensive than
[23:03] (Irbis_) Ethernet, but they increase dramatically the overall performance of our cluster, and that's why they are strongly recommended.
[23:04] (Irbis_) Ethernet, but they increase dramatically the overall performance of our cluster, and that's why they are strongly recommended.
[23:04] (Irbis_) Today, best solution is Myricom net, but tomorrow may vary. Anyway, we must to renember that the PC bus clock may be a bottleneck, limiting the
[23:04] (Irbis_) speed to 133Mbps -33MHz busspeed, today nonwritten standard-. IBM NetInfinity 7000 server series can run four times faster, arise to a 64 bits,
[23:04] (Irbis_) 66MHz PCI bus, but not all cards are able to afford this.
[23:04] (Irbis_) Myricom
[23:04] (Irbis_) Myricom cards and switches that interconnect at speeds of up to 1.28 Gbps in each direction.
[23:04] (Irbis_) It has two different forms: copper-based and optical.
[23:04] (Irbis_) The copper version for LANs can communicate at full speed at a distance of 10 feet but can operate at half that speed at distances of up to 60
[23:05] (Irbis_) feet.
[23:05] (Irbis_) The fiber version can operate at full speed up to 6.25 miles on single-mode fiber, or about 340 feet on multimode fiber.
[23:05] (Irbis_) It has some different network configurations: direct point to point, hub-based, or switch-based.
[23:05] (Irbis_) Its average latency between two directly connected nodes is 5 to 18 microseconds, a magnitude or more faster than Ethernet.
[23:05] (Irbis_) Its switches can be connected in cascade, and this only increments the latency between segments.
[23:05] (Irbis_) Giganet
[23:05] (Irbis_) Giganet network was promoted by Intel.
[23:05] (Irbis_) It uses its own network protocol, discarding IP, but this is transparent to the software.
[23:05] (Irbis_) It is not WAN routable.
[23:05] (Irbis_) It is able to do 1 Gbps unidirectional communications between the nodes at minimum latencies of 7 microseconds.
[23:06] (Irbis_) IEEE SCI
[23:06] (Irbis_) IEEE SCI has a ring-topology-based networking system unlike the star topology of Ethernet.
[23:06] (Irbis_) It is faster to large-scale communication.
[23:06] (Irbis_) It can be used with a torus-2D or a torus-3D topology.
[23:06] (Irbis_) It is the best choose for high number of nodes, because it can support hundreds or thousands of nodes.
[23:06] (Irbis_) It has low latencies (under 2.5 microseconds), and it can run at 400 MB per second (3.2 Gbps) in each direction.
[23:06] (Irbis_) APIs for Beowulf and Mosix clustering programming
[23:06] (Irbis_) Nearly all parallel APIs and communications APIs can be used on Linux. Anyway, the most popular are the Sockets interface, PVM and MPI. The
[23:07] (Irbis_) three APIs can be used as on Beowulf clusters as on Mosix clusters.
[23:07] (Irbis_) Really, on Mosix clusters, we can use nearly all communications API, due to the ``fork and forget'' programming model. If we do a heavy use of
[23:07] (Irbis_) ``/proc/mosix'' filesystem, we can take the best of our Mosix clusters. Anyway, at this time it is used a more general approachment: using standard
[23:07] (Irbis_) parallel APIs to alow compatibility with Beowulfs and COWs -Clusters of Workstations-.
[23:07] (Irbis_) Sockets
[23:07] (Irbis_) The sockets are the faster API. But, unfortunately it does not exist a sockets port to Fortran. It is fully portable between Unix platforms; but it has
[23:07] (Irbis_) incompatibilities between Unix sockets and Windows sockets and problems with data encoding between different architectures. Anyway, pairs
[23:07] (Irbis_) sprintf and sscanf can save the day, sending in ASCII 7 bits, because ASCII is now a standard.
[23:08] (Irbis_) PVM -- Parallel Virtual Machine
[23:08] (Irbis_) PVM has an excellent Linux port, and is fully standard. It is also the non-official parallel programming standard. It can be used in the most of the
[23:08] (Irbis_) languages, including Fortran. Maybe PVM is somewhat slow, but it is easy to learn and use. All programs that use PVM are directly executables in
[23:08] (Irbis_) parallel fashion in Beowulf and Mosix clusters.
[23:08] (Irbis_) MPI -- Message Passing Interface
[23:08] (Irbis_) MPI is the official industry standard. Unfortunately, there is not a full free implementation of the standard for Linux. There are some partial
[23:08] (Irbis_) implementations for Linux, each one with its strong points and weakness. If you find a MPI lib that works fine with your application, OK. If you
[23:08] (Irbis_) don't, you are in a problem. MPI has an specially problematic threaded MPI support. At the time this paper was written, it had no free support.
[23:09] (Irbis_) Fork
[23:09] (Irbis_) Mosix allows to avoid headaches, as one of its stronger points. It is the so-called: "fork-and-foget" model. Basically: Create new tasks with fork(),
[23:09] (Irbis_) communicate them by files on a common share filesystem and enjoy! This only works on Beowulf clusters, not on Mosix clusters.
[23:09] (Irbis_) Linux Virtual Server
[23:09] (Irbis_) The Linux Virtual Server project has implemented a number of kernel patches that create a network load-balancing system for incoming TCP/IP
[23:09] (Irbis_) traffic. The Linux Virtual Server software examines incoming traffic, and based upon a load-balancing algorithm, redirects that to a set of servers
[23:09] (Irbis_) acting as a cluster. That allows network applications such as Web servers to run on a cluster of nodes to support a greater number of users.
[23:09] (Irbis_) On a certain way, it is completly complementary to the Mosix model. Meanwhile Mosix migration is based on process migrations, and sockets can
[23:10] (Irbis_) not migrate, on a Linux Virtual Server only can migrate the socket.
[23:10] (Irbis_) It is important that all Linux Virtual Server must be attached at the same segments. If we want to use more complex topologies, as multilevel
[23:10] (Irbis_) topologies or hive topologies, we will need tunneling IP packets. But sometimes we will need that kind of topologies due to the scalability limit of
[23:10] (Irbis_) Ethernet networks, thus it can be a non-trivial issue making a medium-sized Linux Virtual Server cluster with Linux Virtual Server.
[23:10] (Irbis_) TurboLinux, TurboCluster and enFuzion
[23:10] (Irbis_) TurboLinux has a product called TurboCluster that was originally based on the kernel patches developed by the Linux Virtual Server project. That
[23:10] (Irbis_) means that it allows most of the same benefits, and it has the same drawbacks as the originating project. But TurboLinux has also developed some
[23:10] (Irbis_) tools for monitoring the behavior of the cluster help to manage the cluster. It also has commertial support, that Linux Virtual Server hasn't.
[23:11] (Irbis_) EnFuzion is an upcoming scientific clustering product from TurboLinux that isn't based on Beowulf. It is suposed that it does support hundreds of
[23:11] (Irbis_) nodes, runs any existing software, and it doesn't need custom parallel applications written to the environment. It supports automated load balancing
[23:11] (Irbis_) and resource sharing between the nodes and failed jobs can be automatically rescheduled. It is NOT free.
[23:11] (irbis) Aviso: turboCluster y turboLinux NO son libres.
[23:11] (Irbis_) Linux-HA Project
[23:11] (Irbis_) The objetive of Linux-HA project is not related with neither time of response nor velocity. They are completly different of the two before. The key
[23:11] (Irbis_) is: whatever happends, keep the system running. They are used when is critical that the machine runs, and the machine must resist to hardware and
[23:11] (Irbis_) software fall-offs. It is the general case of servers, but more specificaly of web servers, internet service providers, and in science on computers that
[23:11] (Irbis_) control experimental machineries.
[23:11] (Irbis_) The project has software that can maintain heartbeats between nodes and take over IP addresses of failed nodes. Should a node fail, it uses the Fake
[23:12] (Irbis_) Redundant IP software package to add the address of the failed node onto a working node to assume its responsibilities. Thus the failed node can be
[23:12] (Irbis_) replaced automatically in a matter of milliseconds. For practical use, that heartbeat is usually kept in the several seconds range, unless you have a
[23:12] (Irbis_) dedicated network link between the nodes. At that point, the user applications on the failed system still need to be restarted on the new node. It is
[23:12] (Irbis_) also full supported by a firm.
[23:12] (Irbis_) Some stravaganza
[23:12] (Irbis_) Here we can find some of the weird APIs and products that can be found on the market. At this time there not exist any real aplication over them,
[23:12] (Irbis_) but some day maybe we can find an aplication that uses some of them. Or maybe cann't.
[23:12] (Irbis_) Java
[23:13] (Irbis_) There exists some Java material, very interesting for theoretical computing science, but not very usefull for number crushing problems. Maybe next
[23:13] (Irbis_) century...
[23:13] (Irbis_) Cplant
[23:13] (Irbis_) A new proposal of Sandia labs. Uses new protocols for nearly all, to break the limits of TCP/IP and related protocols. Only for Alpha, only using
[23:13] (Irbis_) portals, but can break the TeraFLOP barrier. At his moment, it is not a broadly extended project due to the cost of the cluster and that there is not
[23:13] (Irbis_) free aplications for it. Old aplications do not work over Cplant.
[23:13] (Irbis_) Legion
[23:14] (Irbis_) Metamodel of paralelism. At this time, there is not any aplication that uses it. Legion is an attempt to build a true multicomputer system. That is a
[23:14] (Irbis_) cluster whereby each node is an independent system, but the overall system appears to the user as a single computer, as Mosix inteeds; but Legion
[23:14] (Irbis_) was designed to support a single worldwide computer consisting of many hosts and many software objects. Within Legion, users can construct their
[23:14] (Irbis_) own collaborative groups. On its theoretical side, Legion provides high-performance parallelism, load-balancing, distributed data management,
[23:14] (Irbis_) and fault-tolerance. It supports high-availability through its fault-tolerance management and dynamic reconfiguration across the member nodes.
[23:14] (Irbis_) It also has an extensible core that can be dynamically replaced or upgraded over time as new advancements or developments arise. The system is
[23:14] (Irbis_) not under a single monolithic control but can be managed by any number of organizations, each supporting an autonomous part of the whole. The
[23:14] (Irbis_) Legion API provides high-performance computing through its built-in parallelism.
[23:14] (Irbis_) All this stuff rely over the user's computer operating system and negotiates between the local and the distributed resources. It handles resource
[23:14] (Irbis_) scheduling and security automatically and also manages a context space to describe and access any object out of all of the possibilities in the entire
[23:14] (Irbis_) system. However, it does not need to run under system administrator privileges on each node and can work within the confines of a nonprivileged
[23:15] (Irbis_) user account. That increases the flexibility of the nodes and users that can join Legion.
[23:15] (Irbis_) This is what Legion team says. At this time, there is not any application, neither real nor toy, over legion.
[23:15] (Irbis_) PAPERS
[23:15] (Irbis_) As part of a project of MIT, they try to show us that the most of the cost of communication was sincronization, not sending information. PAPERS
[23:15] (Irbis_) was build over parallel port interface to show this. It looks that works, but I haven't seen any real aplication over PAPERS.
[23:15] (Irbis_) Portals
[23:15] (Irbis_) A interesting proponsal, strongly defended by SANDIA people. It tries to break TCP/IP and PVM limitations on scalability. At this time, it works
[23:16] (Irbis_) in some interesting proyects at SANDIA. Neither of them are Free Software, then we will have to wait and see. It should be peer-reviewd before we
[23:16] (Irbis_) could evaluate it.
[23:16] (irbis) Tampoco es Open Source, por lo que no he podido ni echar un ojo al codigo fuente. :-(
[23:16] (Irbis_) Unify
[23:16] (Irbis_) Wrappers of MPI over PVM. The power of this stuff is that we can use MPI and PVM and the same time. It is somewhat old, and it is being
[23:16] (Irbis_) superated by another proponsals before it had been used on a real aplication.
[23:16] (Irbis_) Caveats about Linux clustering...
[23:16] (Irbis_) ...or when is better to buy an Alpha.
[23:17] (Irbis_) We will have a lot of work in the case of MPI for some aplications, because it lack of full implementation of the standard. The solution,
[23:17] (Irbis_) unfortunately, is to by a propietary implementation MPI that implements all the protocol (there is one).
[23:17] (Irbis_) Anyway, we would do better buying a huge Alpha SMP or an O2000 than a Linux cluster if:
[23:17] (Irbis_) We have the money.
[23:17] (Irbis_) We have a serial -non parallel- aplication.
[23:17] (Irbis_) None of the parallel aplications equivalent to our serial aplication is good for us. (strange, but it can happend).
[23:17] (Irbis_) The time of parallelizing and executing our aplication will be greater than importation of an Alpha and execution.
[23:17] (irbis) Se aaaaaacabo el duro. Preguntas?
[23:18] (trusmis) sii
[23:18] (trusmis) cuanto vale (aprox) las redes miricom?
[23:18] (toomany) naaaasss
[23:18] ) plas plas plas plas plas plas plas plas plas plas
[23:18] ) plas plas plas plas plas plas plas plas plas plas
[23:18] ) plas plas plas plas plas plas plas plas plas plas
[23:18] ) plas plas plas plas plas plas plas plas plas plas
[23:18] ) plas plas plas plas plas plas plas plas plas plas
[23:19] ) plas plas plas plas plas plas plas plas plas plas
[23:19] ) plas plas plas plas plas plas plas plas plas plas
[23:19] ) plas plas plas plas plas plas plas plas plas plas
[23:19] (viper) plasplasplasplasplasplasplasplasplasplasplasplasplasplas
[23:19] (Fernando) plasplasplasplasplas
[23:19] (Fernando) plasplasplasplasplas
[23:19] (Fernando) plasplasplasplasplas
[23:19] (toomany) :)
[23:20] (carcoco) plas plas plas
[23:20] (Fernando) plasplasplasplasplas
[23:20] (carcoco) plas plas plas
[23:20] (carcoco) plas plas plas
[23:20] (toomany) (todavía no ha acabado la conferencia?)
[23:20] (Fernando) plasplasplasplasplas
[23:20] (Fernando) plasplasplasplasplas
[23:20] (Arthur) clap clap clap
[23:20] (Fernando) justo ahora toomany
[23:20] (mjjje) plas plas plas plas plas plas plas plas plas plas
[23:20] (mjjje) clap clap clap clap clap clap clap clap clap clap
[23:20] (mjjje) plas plas plas plas plas plas plas plas plas plas
[23:20] (mjjje) clap clap clap clap clap clap clap clap clap clap
[23:20] (viper) clopclopclopclopclopclopclopclopclopclopclopclop
[23:20] (mjjje) plas plas plas plas plas plas plas plas plas plas
[23:20] (mjjje) clap clap clap clap clap clap clap clap clap clap
[23:20] (toomany) jueerrrr
[23:20] (Alma) clap clap clap
[23:20] (Fernando) Se abre ahlora el turno de preguntas....
[23:21] (trusmis) cuanto vale (aprox) las redes miricom?
[23:22] (irbis) Lo primero, tienes mucha info de miricom en http://www.inf.ufrgs.br/procpar/disc/cmp134/trabs/T1/001/myrinet/introducao.html
[23:22] (Arthur) Que beneficios se esperan de estas redes ?
[23:23] (irbis) Sobre cuanto cuestan, ahora mismo no tengo seguridad. Puede que unos U$300 por nodo, aunque varia mucho de pais a pais.
[23:23] (irbis) Ademas, habitualmente tienes que meter un Switch
[23:23] (irbis) Sobre los beneficios, muy grandes.
[23:23] (irbis) Practicamente termina con el cueyo de botella de la red.
[23:24] (irbis) Ahora, a cambio, es muy caro.
[23:24] (irbis) En mi cluster yo uso dos tarjetas Ethernet 100
[23:24] (irbis) por nodo.
[23:24] (Alma) entonces, con estas redes no habra congestionamientos?, vale la pena el costo ?
[23:24] (irbis) Una para los paquetes NFS,
[23:24] (trusmis) tienes comparativas o sabes donde las hay entre todo tipo de estos sistemas?
[23:24] (irbis) y otra para la comunicacion del Sw.
[23:24] (trusmis) o SMP VS clustering?
[23:25] (irbis) Uno a uno...
[23:25] (irbis) Alma, en una red de gigahercios puede haber congestionamentos,
[23:25] (irbis) pero si el numero de nodos es alto o la granularida de la aplicacion es muy fina.
[23:26] (irbis) De cualquier forma, aguantan una o dos veces -segun la red- en orden de magnitud mas carga que las placas "avioncito feliz".
[23:26] (irbis) A respecto de SMP o clustering,
[23:26] (irbis) es una pregunta REALMENTE interesante.
[23:26] (irbis) Ahora te estoy escribiendo con un dual Pentium, por ejemplo
[23:26] (irbis) y, por otro lado, uso un cluster.
[23:27] (irbis) SMP es maravilloso, solo que muy caro y poco escalable
[23:27] (irbis) el cluster es mas barato, y mas escalable, pero su rendimiento de paralelizacion flops/procs
[23:27] (trusmis) ok, me interesa alguna grafiquilla o algo así
[23:27] (irbis) es mucho peor.
[23:28] (irbis) Si tienes pelas, SMP es maravilloso para servidores Web,
[23:28] (toomany) osea, que un desarrollo "juntando" ambos, puede suplir los vacios de cada uno...
[23:28] (irbis) o cosas asi. Puedes hasta tener un cluster de maquinas SMP.
[23:28] (irbis) Lo que pasa es que cuando no tienes dinero para mucha maquina, no sueles tener dinero para una placa SMP, que son mucho mas caras.
[23:29] (toomany) bueno, suplir suplir... pero bueno...
[23:29] (irbis) Respecto a graficas, no me he molestado en hacerlas.
[23:29] (carcoco) Existe algun cluster Beowoff en Espanya ???
[23:29] (irbis) La razon es muy sencilla,
[23:29] (trusmis) irbis: y que tal ves los clusteres para servidores web?
[23:29] (trusmis) o valores algo asi
[23:29] (irbis) mi dual pentium no es capaz de tirar con un refinamiento de una proteina de 800 aminoacidos.
[23:29] (irbis) Ahora, para tareas mas simples si es interesante.
[23:29] (toomany) en la Escuela de Ingeniería de Terrassa (Barcelona), tengo un colegita que montó un beowulf.
[23:30] (irbis) Que sepa, algunos grupos estan montando Beowulfs.
[23:30] (irbis) Yo intente montar uno con la gente de Aiken en la UMA, solo que no conseguimos finalmente las maquinas
[23:30] (Fernando) ese tipo de calculos son altamente paralelizables? hay poca comunicacion ??
[23:30] (irbis) termine montando un Mosix en Brasil.
[23:30] (toomany) he oído hablar muy bien de Mosix...
[23:31] (irbis) Depende del tipo de calculo. Dime el calculo, y yo te digo si lo es o no lo es.
[23:31] (toomany) dicen que supera a Beowulf...
[23:31] (Fernando) los que tu haces, los de las proteinas
[23:31] (irbis) Si, Mosix es muy bueno. Y ES GPL. He leido a Linus Torvalds, Alan Cox y Stallman protestar porque Mosix no es libre.
[23:31] (irbis) Ahora, es GPL v2.0.
[23:32] (toomany) :)
[23:32] (irbis) No he conseguido contactar ni con Linux ni con Cox para decirles que SI es libre,
[23:32] (irbis) si lo he hecho con Stallman.
[23:32] (Fernando) habla con mjesus, ella consiguio hablar con cox :)
[23:32] (irbis) aunque la sensacion que me dio es que le importaba bastante poco.
[23:32] (toomany) parece ser que Stallman es el más "abierto" a recibir emilios del "pueblo llano"...
[23:33] (toomany) si... bueno... es Stallman... ya sabes...
[23:33] (irbis) Es el unico abierto. ;-(
[23:33] (Borja) No son GPL? no me lo puedo creer! ;-)
[23:33] (trusmis) irbis: y que tal ves los clusteres para servidores web?
[23:33] (irbis) Pues eso, por mucho que se quejen, MOSIX ES GPL.
[23:33] (toomany) qué ventajas se pueden obtener poniendo un servidor web "clusterizado"?????
[23:33] (irbis) Respecto a los clusters para servidores Web
[23:34] (irbis) debes hacer un estudio detallado del cueyo de botella de tu servidor.
[23:34] (irbis) Si tienes un monton de CGIs y toda la pesca, mejor Mosix
[23:34] (Borja) Cosa en la que por el momento Linux cojea bastante. Ha salido algún buen paquete de monitorización de prestaciones?
[23:34] (mike) a que hora empieza la conferencia...?
[23:34] (irbis) Si tienes mucha carga, Linux Virtual Server
[23:35] (irbis) las dos soluciones son GPL (por tanto libres)
[23:35] (toomany) de dónde se puede pillar?
[23:35] (Borja) (Que de todas maneras necesitaría modificaciones en el kernel y muchos drivers)
[23:35] (pask) linux virtual server tiene un gran rendimiento ..
[23:35] (Fernando) mike hace exactamente hora y media
[23:35] (trusmis) www.mosix.org
[23:35] (pask) y no necesita drivers ....
[23:35] (pask) ,-)
[23:35] (irbis) Mezclar las dos es simple si usas un front-end de maquinas LVS,
[23:35] (irbis) y mandan los cgi a un cluster Mosix por rsh
[23:35] (irbis) Con respecto a lo util que sea,
[23:35] (irbis) es MUY util,
[23:35] (pask) uncluster con LVS con ldirector ....
[23:35] (trusmis) Borja: el top :)
[23:35] (toomany) explica explica...
[23:36] (pask) maravilloso ..
[23:36] (pask) failover y demas ..
[23:36] (irbis) y supone ahorrase una PASTA GANSA a los proveedores de acceso a internet.
[23:36] (toomany) podrías explicármelo, por favor?
[23:36] (irbis) Recueden que tambien tenemos el HA-Linux,
[23:36] (pask) kimberlite too
[23:36] (irbis) y, ademas, tenemos excelentes sistemas "journaled"
[23:36] (carcoco) Google, segun creo funciona a base de un cluster de 2000 maquina linux, no ???
[23:36] (toomany) sip
[23:36] (irbis) yo uso el ReiserFS en entorno de produccion hace unos cuatro meses,
[23:37] (Borja) No es lo mismo, trusmis. Hablo de estadísticas sobre el tiempo que llevan los accesos a disco, cantidad de trabajos en cola de disco, etc.
[23:37] (irbis) ininterrumpidamente en una docena de maquinas, y va maravilloso.
[23:37] (pask) se
[23:37] (irbis) El unico problemas es que meter simultaneamente los parches Mosix y ReiserFS es un poco lioso, pero con truco se puede hacer.
[23:38] (irbis) Ahora mismo, mi consejo para entornos que la carga sea ligada a la CPU es Mosix+ReiserFS
[23:38] (toomany) algunos consejos para que "no se queme el pavo en el horno"???
[23:38] (toomany) osea
[23:38] (irbis) y para entornos que la carga sea ligada a la red LVS+ReiserFS
[23:38] (toomany) para lo que comentabas?
[23:38] (irbis) Tenemos fiabilidad y equilibrado automatico de carga =) pocos dolores de cabeza.
[23:39] (irbis) Alguna pregunta mas?
[23:39] (carcoco) Google, segun creo funciona a base de un cluster de 2000 maquina linux, no ???
[23:39] (irbis) No lo se.
[23:40] (trusmis) conoces algo llamado Scylt?
[23:40] (pask) alguna URL que compare erendimientos de LVS con maquinas "comerciales?
[23:40] (irbis) Pero 2000 es un numero muy grande para un cluster Linux.
[23:40] (pask) carcoco .. eso dicen las "malas lenguas"
[23:40] (irbis) Que sepa, tanto nodo tienen apenas unas pocas maquinas de IBM,
[23:40] (toomany) por mi parte me retiro. Muchas gracias por lo interesante de la charla (aunque sólo haya podido pillar los "ruegos y preguntas").
[23:40] (toomany) Buenas noches a todos/as.
[23:40] (trusmis) pask: openprojets.net esta montado con LVS, alli explican algo creo
[23:40] (irbis) y los trabajos de SANDIA con portals, pro el problema de escalamiento de TCP/IP
[23:41] (irbis) de cualquier forma, puedo equivocarme.
[23:41] (pask) openprojects.net ? ... ubmmmm
[23:41] (pask) lo mirare
[23:41] (Borja) Conocéis Eddieware?
[23:41] (Borja) http://www.eddieware.org
[23:41] (carcoco) no, que es eddiware
[23:41] (carcoco) no, que es eddiware ????????
[23:42] (Borja) Un repartidor de carga para WWW desarrollado por Ericsson, gratuito
[23:43] (pask) ?
[23:43] (pask) que corre sobre ?
[23:43] (pask) solaris ? linux ?
[23:43] (pask) freBSD ?
[23:43] (Borja) Está hecho en Erlang, así que al menos corre en FreeBSD, Linux y Solaris
[23:43] (trusmis) algun lugar donde corran programas paralelizados para ver performance de los clusteres?
[23:43] (Fernando) la web esa no va
[23:44] (Lady) Hola a todos!!
[23:44] (Lady) Hola oroz!!
[23:44] (pask) erlang ... bonita uindad ...
[23:44] (pask) ok
[23:45] (irbis) PReguntas?
[23:45] ) a ver, el que quiera preguntar que ponga una ? en el canal, y se le
[23:45] ) cuando toque. Al acabar de escribir, pongan la palabra fin
[23:45] ) mike
[23:45] ) tu querias preguntar
[23:45] (Borja) Pero no me contesta el www de Eddieware ahora
[23:45] (Fernando) no borja
[23:46] (Fernando) seguro que habrá más preguntsa
[23:47] (Fernando) pero este es un momento perfecto
[23:47] (Fernando) para agradecer al conferenciante
[23:47] (Fernando) su presencia y su interés por el congreso
[23:47] (Fernando) y por supuesto, por su interesantísima presentacion
[23:47] (Fernando) gracias David
[23:47] ) plas plas plas plas plas plas plas plas plas plas
[23:47] ) clap clap clap clap clap clap clap clap clap clap
[23:47] ) plas plas plas plas plas plas plas plas plas plas
[23:47] ) clap clap clap clap clap clap clap clap clap clap
[23:47] ) plas plas plas plas plas plas plas plas plas plas
[23:47] ) clap clap clap clap clap clap clap clap clap clap
[23:47] ) plas plas plas plas plas plas plas plas plas plas
[23:47] ) clap clap clap clap clap clap clap clap clap clap
[23:47] (irbis) Gracias a vosotros por la oportunidad de hablar. :-)
[23:48] (carcoco) plas plas plas
[23:48] (pask) gracias ...
[23:48] (viper) plas plas plas plas plas plas plas plas plas plas plas
[23:48] ) plas plas plas plas plas plas plas plas plas plas
[23:48] ) clap clap clap clap clap clap clap clap clap clap
[23:48] ) plas plas plas plas plas plas plas plas plas plas
[23:48] ) clap clap clap clap clap clap clap clap clap clap
[23:48] ) plas plas plas plas plas plas plas plas plas plas
[23:48] ) clap clap clap clap clap clap clap clap clap clap
[23:48] ) plas plas plas plas plas plas plas plas plas plas
[23:48] (Borja) plas plas plas plas plas plas plas plas
[23:49] ) habra mmas no ?
[23:49] (Fernando) si mjesus
[23:49] (Fernando) como dijimos antes
[23:49] (Fernando) mañana intervendrá
[23:49] (Fernando) Borja Marcos de la Puerta
[23:49] ) plas plas plas plas plas plas plas plas plas plas
[23:49] ) plas plas plas plas plas plas plas plas plas plas
[23:49] ) plas plas plas plas plas plas plas plas plas plas
[23:49] ) plas plas plas plas plas plas plas plas plas plas
[23:49] (Fernando) con una conferencia sobre temas de seguridad
[23:49] (Fernando) titualada
[23:49] (carcoco) A que hora ????
[23:50] (Fernando) Los enanitos del bosque han desconfigurado mi sistema
[23:50] (irbis) :-)
[23:50] (Fernando) 19:00
[23:50] (viper) jajajaja. Que loco!!!
[23:50] ) plas plas plas plas plas plas plas plas plas plas
[23:50] ) clap clap clap clap clap clap clap clap clap clap
[23:50] ) plas plas plas plas plas plas plas plas plas plas
[23:50] ) clap clap clap clap clap clap clap clap clap clap
[23:50] ) plas plas plas plas plas plas plas plas plas plas
[23:50] * MJesus quiere saber si el server del buscador google es alguna suerte de Mosix
[23:50] (irbis) No lo se. Sospecho que es un Beowulf, los Mosix son extranhamente poco conocidos.
[23:51] (trusmis) irbis: no te pierdas la del 12-12
[23:52] (Fernando) tan pronto como sea posible, colocaremos la 'grabación' de esta presentacion en nuestra web
[23:52] (irbis) Cual es?
[23:52] (Fernando) asi como la documentacion aportada por el conferenciante
[23:52] (Fernando) umeet.uninet.edu/
[23:52] (Fernando) para poder leerla con tranquilidad
[23:52] ) eso....... la del 12 de diciembre va a ser maravillosa irbis
[23:52] (Fernando) umeet.uninet.edu/spanish/des.html
[23:53] (irbis) Ya veo, :-)
[23:53] (irbis) Es de mosix. :-)
[23:53] ) plas plas plas plas plas plas plas plas plas plas
[23:53] ) plas plas plas plas plas plas plas plas plas plas
[23:53] ) plas plas plas plas plas plas plas plas plas plas
[23:53] (viper) Fernando: pueden ponerse las consultas en su totalidad, porque las que estan dice; y seguimos por un rato...
[23:53] ) plas plas plas plas plas plas plas plas plas plas
[23:53] ) claro, pero desde que se cierra, ya no se publica
[23:53] ) es como cuando se acaba el partidode futbol
[23:53] (Fernando) viper eso se pone cuando de lo que se lee en el canal, no se puede sacar nada en claro ;)
[23:54] (Fernando) por ejemplo ahora llevamos dos o tres pantallas asi
[23:54] ) que ya no se transmite mas aunque las peñas salgan de cale
[23:54] (trusmis) irbis: espero que te guste y que sigamos en contacto
[23:54] (viper) si. jajaja
[23:54] (irbis) Usease, ahora estan las camaras desligadas?
[23:54] (Fernando) no exactamente irbis
[23:54] (Fernando) luego hacemos 'limpieza' a mano

AQUI EL ENCARGADO DE LA LIMPIEZA A MANO :P y no pienso quitar nada de lo que se diga XDDD :P LO QUE SIGUE ES INTERESANTE SEGUIR LEYENDO DESPUES DE LAS BROMAS

[23:55] (irbis) Pena, porque pensaba decir la verdad:
[23:55] (Fernando) XD
[23:55] (Fernando) dila
[23:55] (irbis) TODO LO ANTERIOR ES MENTIRA!!!!
[23:55] (Fernando) siempre podemos cortar lo que quieras
[23:55] (Fernando) XD
[23:55] ) plas plas plas plas plas plas plas plas plas plas
[23:55] ) plas plas plas plas plas plas plas plas plas plas
[23:55] ) plas plas plas plas plas plas plas plas plas plas
[23:55] ) plas plas plas plas plas plas plas plas plas plas
[23:55] (oroz) XD
[23:55] (pask) viva microsoft!!!!!! el windows 30000000 nos salvara la vida ... !!!!!!
[23:56] (irbis) SEGUN LA SERIA MINDCRAFT, UN 486 CON NT SUPERA EN RENDIMIENTO A UN CUSTER DE 256 DIGITAL CON LINUX!!!!
[23:56] (pask) juas juas juas
[23:56] (carcoco) plas plas plas
[23:56] (carcoco) plas plas plas
[23:56] (carcoco) plas plas plas
[23:56] ) jjajajajajajajjaajjajaç
[23:56] (irbis) ESTOS BENCKMARKS INDEPENDIENTES HAN SIDO PUBLICADOS EN LOS MAS IMPORTANTES SITES DE LA WEB!!!!
[23:57] (irbis) www.microsoft.com, www.msm.com y www.ilovegates.oeg!!!!
[23:57] (irbis) X-DDDDDDDD
[23:57] (carcoco) Una pregunta: ? ...
[23:57] (irbis) Si
[23:57] (Fernando) mejor sera
[23:57] (Fernando) XD
[23:57] (carcoco) Ademas del fortran: Se pueden usar otros lenguajes de programacion.
[23:58] (irbis) RECOMIENDO usar otros lenguajes de programacion.
[23:58] (Fernando) jajajaja
[23:58] (Fernando) bueno fortran, casi emociona
[23:58] (irbis) He tenido la "suerte" de programar en Fortran
[23:58] (Fernando) los orígenes
[23:58] (trusmis) carcoco: c normal y corriente
[23:58] (irbis) es como hacer el camino de santiago con chinchetas en las botas
[23:59] (irbis) puedes ganar el perdon de Dios, pero es una experiencia que no se olvida jamas.
[23:59] (pask) juas juas ...
[23:59] (irbis) Ahora en serio;
[23:59] (carcoco) Y que ocurre si no tenemos el fuente de nuestros programas de procesamiento.
[23:59] (pask) sparse matrix ... blas y todo aquello ...
[23:59] (pask) juas juas
[23:59] (irbis) yo programo en C, y toco algo de C++ porque los wrappers de QT para C estan abandonados
[23:59] (pask) blasm ...
[23:59] (viper) buenos, chicos, todo muy interesante!!!! Buans noches y nos vemos en un proximo encuentro
[23:59] (carcoco) El cluster realiza el reparto del procesamiento de forma transparente ???
[23:59] (irbis) un alumno al que el oriento el proyecto fin de carrera programa en perl
[00:00] (irbis) y no tenemos problema.
[00:00] (irbis) Si usas Mosix, puedes hacer el programa en lo que te de la gana,
[00:00] (irbis) hasta en GnomeBasic.
[00:00] (trusmis) carcoco: si
[00:00] (trusmis) al menos en mosix
[00:00] (irbis) Si usas Beowulf, la cosa cambia.
[00:00] (irbis) Tienes que usar alguna biblioteca de paso de mensajes,
[00:01] (irbis) lo que te ata un poco -solo un poco- mas.
[00:01] (irbis) Ahi el GnomeBasic no te vale,
[00:01] (carcoco) Otra pregunta. El cluster mas pequenyo supongo que seran dos maquinas?
[00:01] (irbis) pero tienes trescientosmilcincuentayun lenguahes con Wrappers de PVM o que puedes enlazar codigo en C de sockets.
[00:02] (trusmis) carcoco: si
[00:02] (irbis) De cualquier forma, usando un sistema de ficheros comun montado por NFS y archivos, puedes hacer virgerias.
[00:02] (trusmis) irbis: como que?
[00:02] (irbis) No necesitas mas nada, y puedes hasta desarrollar aplicaciones en...agggg... Pascal... agggg X-D
[00:03] (irbis) Usando archivos de bloqueo, y archivos para intercambiar datos.
[00:03] (trusmis) y no es mas eficiente semaforos y memoria compartida?
[00:03] (irbis) Si, pero como programas eso, por ejemplo, desde Basic?
[00:04] (Fernando) desde basic no se
[00:04] (trusmis) ah, ok
[00:04] (Fernando) pero en fortran si }:)
[00:04] (irbis) Ademas, memoria compartida en clusters nunca la he visto funcionar como dios manda.
[00:04] (trusmis) irbis: ya , ya lo he notado también
[00:04] (irbis) El paso de mensajes suele ser mas eficiente y comodo (aunque los fanaticos del paso de mensaje me sacudan por esto) :-)
[00:04] (Fernando) es un poco articial, memoria compartida en clusters, no?
[00:05] (trusmis) las señales en mosix tampoco es que sean muy eficientes que digamos
[00:05] (irbis) El problema no es tanto el pasar la informacion de un lado para otro, como el tema de localidad y de cache de informacion.
[00:05] (trusmis) Fernando:con una red de gigabit tiene sentido
[00:05] (Fernando) ya
[00:05] (irbis) En mi problema, por ejemplo, la semantica se adapta mas a memoria compartida que a paso de mensajes.
[00:06] (Fernando) pero entonces empieza a tener mas sentido SMP (tal vez?)
[00:06] (irbis) Sabes lo que hago? Mando mensajes con ORDENES sobre lo que hay que hacer con la memoria (operaciones vectoriales)
[00:06] (irbis) Por partes;
[00:06] (irbis) el problema, tanto con "avioncito feliz" como con Mirinet es el mismo,
[00:07] (irbis) quien tiene la copia master de las paginas comunes, quien tiene la cache, y como se mantiene la consistencia semantica
[00:07] (irbis) en caso de que varios procesos escriban simultaneamente en la misma pagina.
[00:07] (irbis) Necesitas bloquear las paginas de memoria si un proceso accede a ella. No es tanto el problema del broadcast de las paginas sucias,
[00:08] (irbis) sino un problema de bloqueo entre procesos.
[00:08] (trusmis) para eso están los semáforos
[00:08] (irbis) Aunque uses un tamanho variable de zona bloqueada, sueles tener muchas hebras esperando semaforos,
[00:08] (irbis) con lo que pierdes mucho rendimiento en paralelismo.
[00:09] (trusmis) pero no se puede evitar...... a mi modo de verlo
[00:09] (irbis) Por otro lado, el SMP no tiene ese problema, puesto que hay solo una memoria fisica para los procesadores,
[00:09] (irbis) ahora, tienes el problema del escalamiento. No puedes colgar 64 CPUs de un bus PCI.
[00:09] (irbis) Ahi esta el problema, en que no se puede evitar.
[00:09] (Borja) De un PCI no, pero de otros buses (como los UPA de Sun) sí
[00:10] (irbis) No es un problema de velocidad de transferencia de informacion,
[00:10] (irbis) sino de bloqueos entre hebras.
[00:10] (irbis) Tienes toda la razon con la UPA,
[00:10] (irbis) de hecho, la Digital tiene bloques de memoria distribuida que se pueden enlazar a un cluster,
[00:10] (Borja) Es una auténtica pasada de diseño (en mi opinión, al menos)
[00:11] (irbis) de forma de que cada procesador tiene su memoria local y puede leer y esctribir bloques de memoria en la unidad de memoria,
[00:11] (irbis) que es casi tan grande como una CPU.
[00:11] (pask) E10000 rules
[00:11] (irbis) Ahi el tratamiento de los bloqueos se hace de forma local a la memoria, centralizado, por lo que
[00:11] (irbis) no es problematico.
[00:11] (irbis) Ahora, el costo es muy grande, y no siempre compensa.
[00:13] (MJesus) 1 millon ?
[00:13] (MJesus) de pts ?
[00:14] (Borja) ;-)
[00:14] (MJesus) menos ?
[00:14] (Borja) Una E10000 en configuración básica costaba antes pr lo menos un millón de *dólares*
[00:14] (Borja) (Y eso creo que con 16 CPUs; acepta hasta 64)
[00:14] * MJesus traga saliva.....
[00:14] (pask) vale menos ..
[00:14] (Borja) ha bajado?
[00:14] (pask) un momento ....
[00:14] (Borja) Ah, manejo el precio de cuando la sacaron.
[00:14] (pask) creo .. que vale menos ..
Y seguimos un buen rato más charlando...

Contact: