Cafe Bucky

I’m going to shelve my work on MacSimStick for the time being.  I’d been hoping to crack the nut of running Mono and everything else off of a disk image (Apache, PHP, and MySQL run fine, as does OpenSim paired with a Mono install on a local disk).  It doesn’t seem like the need is there, and I’ve been spending hours trying to get install_name_tool and dynamic libraries to play nice.

Sometimes it is good to fail fast and move on.

So anyways, there is a distribution out there that will run, provided you have Mono already installed.   Maybe that is a success and people are happy with that.  I’ll turn my attention to OpenSim paired with Amazon’s EC2, and Unity.  If someone makes progress on MacSimStick, and there’s interest, I may jump back in – I do have several ideas / goals for it.

I am happy to announce the availabilty of MacSimStick, Alpha 1 (Pioneer Possums).

The goal of the MacSimStick project is to have a self contained OpenSim + MAMP environment for Mac users to run off a USB key. It is usable now for those who already have Mono and possibly X11 running on their Mac. I have not yet solved the step of getting Mono to run off of a USB key.

So, this release is called “Pioneer Possums”. If you have a working Mono environment, chances are good that this will work for you. I am looking for the Pioneers that will check this out. Hopefully one of them will be able to help me solve the Mono configuration aspect.

The release is distributed as a Mac disk image.  Download the compressed image – it is roughly 370  meg.  Uncompress the image, double click it to mount, and run “Start.sh”.

There is a README file in the top level that gives details as to the Test User, ports, and the password.  There is also a webserver specific to the project at localhost:8080.

I want to say thanks to the folks that given me encouragement, and who give this a go. I could have stumbled away in isolation trying to get the first release perfect, but I think it is more important to get a working snapshot out to the community – it will hopefully speed up the progress towards the goal of being truely self-contained. Believe me when I say I cant wait to get this out there in a more polished, ready to go form.

I may follow up with a much more techy/detailed posting, but I wanted to keep this one pretty brief and straightforward: “if you are on a Mac, use OpenSim, and have Mono set up, give this a go”.

Cheers!

As I have recently written, I’m working on getting a self contained environment together for running OpenSim and LAMP services on a Mac.

In other words, running a virtual world on your Mac off of a USB key.

So far, so good. But … I’m stuck!

I need some help in getting OpenSim and Mono configured. I’m running into a showstopper problem when a user logs in from a viewer. I’ve decided to put up a snapshot of what I have so that some experienced eyes can take a look at the problem.

The Pre-Alpha snapshot is about 240 meg, and you can get it right now.

The snapshot is a Mac disk image (a compressed sparse bundle). Download, uncompress, and mount. It should mount as /Volumes/MacSimStick. All of the main components are under /Volumes/MacSimStick/usr/local.

See the top level README for ports, passwords, layout, etc.

So what does work?

Apache, PHP, MySQL, and the Diva Wifi interface are all doing fine. There are some configuration tweaks to do, and lots of files and directories that can be removed. The goal right now is to get the end to end functionality sorted, and then we can go back and prune & tune. There is a simple minded shell script, Start.sh, at the top level. I expect that anyone toying around with this at this point is going to be in a terminal window, and is comfy with launching apache, mysql, opensim, etc from a command line.

What is the problem?

The problem seems to be something about how Mono deals with the network. I provided a full logging of the problem in /Volumes/MacSimStick/Logs/OpenSim.log. That file shows the full run of Configure.exe, followed by a session of OpenSim.exe.

Some suggested ways to attack this …

Use your own version of Mono against the diva distro in MacSimStick. If that works, then we know we have a problem specific to my install of Mono 2.8.

Did I miss anything in the Mono or OpenSim config files? (probably!)

Could the problem have something to do with virtual network interfaces (such as the one created by Parallels?)

Is the version of Mono just fine, but we find that it works with an older OpenSim distro?

For reference, my Mac is a 2006 Intel iMac running 10.6.4. I have tried the phoenix and imprudence viewers, and I get the same problem each time. At least that part is consistent.

What are the next goals?

Well, once we have something that functions much like the Windows USB counterpart, I will turn my attention to:

• Simple Startup and Shutdown
• Getting rid of excess files and directories
• Tuning of all config files
• Making use of the web server. Provide info on where to get viewers, a local wiki for the project, and be able to serve as a web gateway for objects in world that need it.
• Getting a good logo

Some more goals:

• Be completely self contained (not touch the local disk at all.. I do currently use a MySQL socket in the standard /tmp location)
• Allow for different mount points besides /Volumes/MacSimStick (am not sure if config files can take care of that for everything…)
• Make it easy to change ports and passwords

I think this is all doable. Although one part of me would love to hold off on snapshots until I have everything working end to end, it’s more important to know when to ask for help! That way, those of us on the Mac side can get going on this that much quicker.

Thanks for any help!

Recently some enthusiasts got together the idea of running OpenSim off a usb key. Even doing a video to show the process.  Pathfinder did a great post on it, as did Ener Hax.  Tateru Nino made a ready to go version of it available.

Pretty Cool, if you are a Windows person.  You plug in a usb key, press a couple of buttons, and have a self contained virtual world on your machine that you can connect to with your fave viewer (SL, Phoenix, Imprudence, etc.)

I’d been waiting to see if someone would do the same thing for the Mac folks.  Time Passed …

… Well … ok, fine, I’ll do it!

So I am undertaking the MacSimStick Project.

I must be out of my mind.  There are a lot of moving parts :)

So What Is The Goal?

The goal is to insert a usb key into a mac, click on an icon, have a self contained virtual worlds environment come to life, use it, and to be able to walk away with any changes you make.

That nicely glosses over about a dozen things happening in the background.

Some assumptions and goals:

• be self contained, keeping off of the local disk as much as possible (log files, etc)
• dont make the user install anything
• wont assume what they have installed (such as Mono).
• dont interfere with web server or database already running on the machine (use different ports)
• provide the power to change things (MySQL and OpenSim consoles), but work without requiring their use

What Are The Puzzle Pieces To This?

• Apache 2.2.17
• MySQL 5.1.52
• PHP 5.5.3
• Mono 2.8
• OpenSim – Diva Distribution

A reason why I include Apache and PHP is because it opens up a lot of interactively options for scripts in a Sim to call into a local web server.

Where Am I With It?

I am working with a disk image on my iMac:  /Volumes/MacSimStick

The first iteration of this will depend on that name.  I compiled Apache, PHP, MySQL, and Mono to depend on paths within /Volumes/MacSimStick/usr/local

Apache, PHP, and MySQL work fine.

The Diva Distro of OpenSim is connecting to MySQL and writing to the database, but bombing out further in the process.  I’m thinking that a good approach will be turn off a lot of things (why is it trying to call JScript?), get things working end to end, and then slowly turn some things back on.

If I am still stuck in a day or so, I will upload what I have to get some more eyes on the problems.

What’s Next?  What Do I Need?

As far as configuration goes:

• corral all of the config files together by having a Config dir full of links to the actual files (PHP, Apache, MySQL, Mono, OpenSim)
• create a startup script which launches a terminal session (using the “screen” command).  This will provide separate shells for MySQL and Opensim.
• create a shutdown script

I need beta testers.  I dont want to make this widely available until I am confident that it works for a handful of folks (for one thing, it’s going to be a sizeable download).  Contact me at my gmail.com address: javajoint, and I will point you at a download.  I am pondering if I want to put this up at Amazon S3 or Dropbox …  It’s going to be a few hundred meg.

I need some extra eyes checking out the configuration.

Donations!  This is going to cost me in hosting, and I am between jobs.  If you can help sponsor the work, cost of hosting, etc, I’d sure appreciate it.  I’ll stick a PayPal button at the bottom of this post.

More on this soon.  Thanks for the enthusiasm so far!

Not long ago, I wondered aloud about what to learn.  Even if I stopped sleeping, I’d still not have the time to absorb all of the things I’m interested in.  I’m glad that I will never be bored, but I am always dealing with the priorities of “what to focus on?”  It’s a fun side of being a generalist: so much is so damn interesting!

So I thought I would write about the flip side, and mention what I am learning.  That way, potential employers and friends will know that I am not sitting around cooking bacon for my cat all day, and that I do have a project or two.

I am focusing on Cloud Computing, C# and .Net/Mono, NoSQL, OpenSim, and Unity.

They all tie together too, kinda.

As you know, a big interest of mine is Virtual Worlds.  I’m a champion of OpenSim and the Hypergrid, because I see a lot of parallels to where we were with Apache and the Web in 1995.  I am glad we have the Windows Sim on a Stick download which utilizes the Diva Distribution.  I am  currently looking into how to use MAMP to do the same thing for the Mac community.

Now that I have mentioned the server side of things, let me do a quick take on the client.  As I’ve written, I really want to see a Unity-based front end to OpenSim.  I have started porting my Slider object from SL to Unity.   My work so far has been in JavaScript, but in order to network it with, say,  Smartfox Server, I’ll be writing a bit in C#.  I see some good reasons to learn C# and Mono (starting within the contexts of Unity and OpenSim [1]).   I’ve been reading Programming C# 4.0.

[1] Seeing as how OpenSim is written in C#

So far so good.  You may be wondering how I tie Cloud Computing into this.  On the Cloud Computing side, I’ve been a user of Amazon S3 for a while.  EC2 was sorta off my radar, but a recent post by Ener Hax caught my attention, where she points at the idea of using the cloud to host OpenSim instances.  This put me on the chain of thought of “how to quickly scale for events in Virtual Worlds?”.  That has me delving into Amazon Web Services, and I am reading Jeff Barr’s Book “Host Your Web Site in the Cloud“.

The last bit of my current techy study stuff is NoSQL.  I’ve been through the try-redis tutorial and love it.  I am decent with MySQL, but I’ve always been a big fan of hashtables/associative arrays/keys & values etc… it fits my mental model of what I like to do in a program (index into an array, based on a key) — having that approach for a database makes sense for a couple of projects I have in mind (my photo database, and ThereThen addresses) .

So that’s my current focus.  I see jobs come across that want other things (like Java…), but I like what I’ve picked for myself!

I was just reading an article on MakeUseOf.com: Create A Google Custom Search Engine For Your Research

I got inspired to tinker with a custom search engine for Virtual Worlds stuff.  Let me know if you want to collaborate on it.

Try a sample search. Go on. You know you wanna ;)

I have been observing Second Life and Linden Lab every day for the last 4 1/2 years (yep, I am Bucky Barkley there).  Around January 2009, I started wondering what event would trigger a migration to OpenSim.   Gwyneth had written a great post back then, which has helped shaped my thinking.

When I saw the announcement this past Monday that Linden Lab would discontinue the 50% discount for educators and non-profits, with less than 3 months notice, I thought “aha! this is the trigger event!”.   There has been much consternation in the SL community lately, but now it has reached the point where a sizable group will take action.  For Linden Lab to have done this to a group of people who have been some of their strongest supporters, is nothing less than a Public Relations disaster.

A migration is in the offing.  Many will be leaving Second Life, and will be heading to OpenSim-based grids.  It creates Opportunity.

You may say “Daniel!  You are overreacting!  Let those educators and non-profits go!  Who needs them!?”…

…And you are entitled to say that :-)

But I say this is something that triggers a whole series of events.  I think that Linden Lab is going to run into some unintended consequences.  They may think that a certain percentage of their user base will migrate, and that it will be an isolated event.

But that is not all that will happen…

When the non-profits and educators go to OpenSim-based grids, they will bring with them a bit of money, expectations, and a lot of drive.  I will quote John Lester (Pathfinder).  He is a former Linden, and in his post “With every Exodus comes Expansion: Educators and Non-profits in Second Life“, he wrote:

nothing drives rapid platform development like a big influx of eager users

Therein Lies Opportunity.  The needs, funds, and energy of the non-profits and educators will speed up the development progress of OpenSim and OpenSim-based grids.  This progress, in turn, will make migrating that much more attractive to other groups that are not directly affected by the pricing increase.  Educators and Non-Profits dont exist in a bubble; they have friends in other communities, and they shop a bit as well.  Their needs, influence, and experiences ripple far beyond their sphere.

I have spent 4 years working on projects in the Info Island Archipelago, and I can say firsthand that there is no clear boundary between the folks there and all of the other communities in SL.  It would appear that LL is taking a gamble that there will be a limited, community specific migration.  They have poked at an anthill, and have not figured just how fast all the sleepy ants within will awake, spring into action, and migrate.

The trick for everyone involved is to get past being upset with Linden Lab, and to work together to get things going in brave new OpenSim worlds.  There is a need for content creators, tool providers, builders, event coordinators, and for many other talents to come along for the ride.  It is a time for all manner of specialists to help each other bootstrap into the post-Second Life era.

So, like I said, Therein Lies Opportunity.   Along with this, those who will need help should be aware of the following:

Not every “solution provider” will be competent.  Not every grid host will be on the level.  Not every Virtual Worlds Consultant will know what they are doing.   Look for proven experience, real names, and always have a Plan B.

As with everything else, do your research before committing time and money.  There are many people right now who are very happy with their projects in OpenSim.  There are failures as well…

…It’s just like the web.  Moving away from the walled garden of SL means that more things need to be considered (who is my host?  am I on the HyperGrid?  How do I find people, places, and things that are scattered across grids?)  But, along with that, there is much more control and reward.  As one example, you can backup your own work, move to another provider, and restore it.

These are pioneering days.  Second Life has served its purpose as an excellent introduction to Virtual Worlds.  They are one chapter in our books.

It’s time to start writing the next chapter, and Therein Lies Opportunity.

Overview

I’m a longtime SL user (March 2006), and I think the 1.x and 2.x codebases are becoming an evolutionary dead-end.  I think the community will face a day of reckoning, and developers will show a way forward.  In some ways, this mirrors the situation of the LL server versus OpenSim.  The Unity development tool would seem to be a very attractive avenue forward.

The server side has been reinvented by OpenSim.  The client side needs a huge reset.

1.x and 2.x Codebases

The current situation with SecondLife / OpenSim viewers is that we have two codebases: 1.x and 2.x. The overwhelming majority of users are on some variant of the 1.x codebase (SL Viewer,
Imprudence, Phoenix).  The 2.x users are, for the most part, on the SL Viewer 2, or on Kirstens Viewer.

Another factor in the current situation is that Linden Lab has put all development effort into 2.x.  They know that the UI issues for the 1.x users are pretty significant.   Since most of the community doesn’t want anything to do with the 2.x interface, Linden Lab will have a hard time attracting much unpaid developer effort to their side.

The other trend affecting LL is that they have some features coming up that will require the 2.x viewer (meshes come to mind).   It’s an unfortunate mix of variables.

For some time to come, the 1.x viewer community will thrive.  They have made significant progress in giving the community what it wants (double click teleports, area search, inventory mirroring, better LSL editing options, more convenient Windlight UI access, just to name a few).  If LL were to pull support for 1.x viewer access [1], it would backfire in a big way — it would be an event that would lead to mass migration.  Whether they like it or not, they are somewhat forced to keep the 1.x viewer community connected.  They have clearly painted themselves into a corner.

[1] nah, they wont — I have a backchannel on this, as well as what was stated on The Developers List and also this one

In short, the 1.x UI is clearly the winner, but on the server side, the way forward will require 2.x.  The LL server / client situation is messy.

… And… neither viewer stream could hope to compete with an AAA level game engine within, say, the next year or two.  My concrete example is that the Unity engine is far ahead in many areas.

Enter the Unity Game Engine

What of true advancements?  What would have to happen to the viewer to bring it up to the standard of a AAA game title?

Looming on the horizon, there are significant walls for the 1.x and 2.x codebase.  To explore but a few areas, what can be done about Graphics, Animation, Sound, and Terrain?

Let’s focus on the positive for a bit :)

Graphics – A quick look at Unity demos (for instance, Tropical Paradise,  Shadows, and Butterfly) gives a strong hint as to its power.  Observe more lights in one scene, dynamic shadows on older hardware.  Custom shaders that can be written by 3rd parties.   Unity 3.0 will be released soon, and goes even further:

• Deferred Rendering — Dark Unity
• Beast Lightmapping — Lightmapping in Unity 3
• Image Effects

Physics – Physics in Unity are very advanced:

• http://unity3d.com/unity/features/physics
• cloth physics in 3.0

Animation – Unity has advanced object and character animation:

• game objects
• locomotion extension
• animation blending GUI

Sound – so much control as to mixing and where the Audio Listener is positioned

Terrain – very powerful from the built in editor, as well as from scripting, and a very well done extension (Terrain Toolkit)

Editing (3.0) – Snapping & Marquee Selection — also note that there is a lot of scripting support (in JavaScript, C#, and Boo) which could be used to develop in-world editing.

Deployment – Here is the Overview for 2.6.  Unity is very well suited for Windows (Direct3D 9), Mac (OpenGL), the Web, Nintendo Wii, and the iPhone.  They are working on deployment for the Android, Xbox 360 and the PS3.

Where To Go From Here

It should be pretty clear that the 1.x and 2.x have little hope of catching up to something like Unity 2.6.  In the year or two it would take to get close, Unity and other game development engines will be that much farther along.  To me, it seems clear that there is a choice:

1. Keep advancing the 1.x and 2.x codebases, picking up some of the features of Unity, but still being limited to Mac, Windows, and Linux
2. Freeze current codebase (Linux will have to stay where it is…), and take the things about SecondLife and add them to an app built with Unity.  This will pick up additional platforms, and will tap into a much larger development community.

I note that there are at least two OpenSim + Unity projects underway:

• Resonance Live
• Heritage Key, by Rezzable

Enjoy the links, ponder it, and think about where you want to put your development efforts for the next 2 years ;)