kazzuya.com

I found a paper on Intel's site. It's from someone at Id Software.
I think it's pretty close to what's behind the cool MegaTexture name.

..basically a very optimized progressive-JPEG-like (edit: actually streams but it's not progressive in the "progressive JPEG" sense) streaming and decompression in real-time. Something very close to what I'm doing recently (as a task in a project, not as a full blown research).

For my implementation I decided to put an extra effort and make it easy to use outside the main application. This means that I'm going hide all my "nice" support classes and types and only expose the bare minimum for anyone to use the proposed functionality.

I set the goal of making a DLL out of this progressive CODEC, but more DLLs for the future.
The reason for a DLL rather than a LIB is that static linking can be pretty tricky. For example I have a global new and delete overload because I normally need 16/64 bytes aligned memory.
It's nice to be able to use a lot of my common library, but it wouldn't be nice to force headers and symbols onto other potential users.

Thinking in a DLL way also makes it easy to write at least one clean class that is easy to understand and document. The class exported in the DLL header is a bare-bones interface with a pointer to an actual hidden implementation class (aka PIMPL).

...basically I'm talking about modularity ! ..back from the dead.. saving the day where the OOP abuse complicates APIs.

cool

Italy lost to Spain in a match for the European Soccer Cup (whatever it's called 8).

On one side I wish Italy won.. on the other side, it's good that Italians understand that they need to get busy and get a culture (practical and scientific, not humanistic !).. fuck soccer ! (Affanculo il calcio !!)

Italians, move your asses ! Kick out the lame politicians and then kick yourself in the butt.. because it's the culture that is wrong.

Change yourself !

wooooo

P.S. Coglioni Del Mondo !

I'm a bit lost. The older (wiser ?) I get the more I realize how everything in life is so unstable and relative.
Most things are what we make of them. You could be stupid and become the president of a big country, start a few wars, change history for better or for worse. People then may or may not realize how stupid you are.. but it will still be a bit win for chaos. Chaos rules modern civilizations, big time.

At work things are going well.. relationships with coworkers improve everyday. As a senior employee, I feel that younger coworkers have certain expectations towards me.. which makes sense, but at the same time one could be wrong on something at any age.
Some people will challenge the status quo more than others. Sometimes I'm challenged myself, though I'm a person that challenges things... challenge the challenger !

You can challenge something, demonstrate the opposite and possibly be successful in your job. Or you can go along and still be successful.

Basically I think that nothing is really well defined. I can see how I could take a path at work and focus on something or focus on something else.. there is a lot of potential everywhere. I could be rich and famous if I cared for that more than other things.. then people complain that they are not rich, but I can't see how it's not their fault.

Same is with personal relationships. A recurring pattern:

10 find a girlfriend
20 take her for granted and get tired of her
30 break up
40 regret for a while
50 find a new girlfriend
60 stay in limbo for a while
70 lose interest for the previous girlfriend
80 goto 20

.. in the meantime there is this ghost.. the ghost of the "M" word.
My opinion is that if I didn't leave Italy, I'd be "M" by now ("M" is pronounced: "fucked !") .. because life would have been much more static. I'm not a wild adventurer, but you don't move to California and then to Tokyo only to get stuck in some gear.
I sometimes wonder how fashion and female customs affect economy. With all those cute girls wearing sexy clothes everyday.. perhaps affects how men think and behave. Perhaps men would be less motivated at work if there wasn't so much "fish in the sea" (though not easy to catch !) constantly teasing the hungry worker.. work harder, get richer, buy me an expensive bag !
And those sexy girls definitely influence other girls.. it's like a race to who's cuter.. or at least it's a reminder that if you don't act cute or dress sexy, then some other girl will and she'll take the better man.

Basically a competitive environment..
Of course all this has to be taken with a grain of salt. I think I see the general rule to get people going and working. It's true everywhere, but perhaps slightly accentuated in Japan. To what degree and to what extent I don't know, but I can definitely see why women in Rome and in Los Angeles dress a lot more casual than their Japanese counterparts.

..but where does all this lead ? I'm supposed to feel arrived, to want to settle, to want to go home early. Instead I want to do my programming, I want to code graphics, mostly, but anything else.. I like the freedom to go about projects at work or at home.. and I find it hard to believe that I will ever want to get into a relationship that is destined to become affection and little or no passion in the best case. But more like fighting, cheating, separation in the more common case.

Every time the "program counter" gets to 50 I'm glad that I'm still in the loop, because every cycle is a bit different, every time I learn something new and I fear the day when I will have to break that cycle.

ummmmmmmmm !!!!

Two days ago I rented 3 movies from the local store. At somebody's request I also rented The Helix... Loaded (Ahorix in Japanese (aho = idiot)) a parody of The Matrix.
Not being a big fan of the original movie and knowing how rare can be a good parody, I rented it with some skepticism.
The movie however beat all my expectations, in a negative way. I couldn't force myself past the first 10 minutes.
This parody is completely pointless, it looks like some home movie, but a lot less funny, because it also includes some decent (by home-movie standard) special effects. At least it could have been funny for being blatantly cheap, but instead it's sad for showing that someone actually put any money into it (and me for accepting to rent this lemon).
I'm just glad that I only wasted 10 minutes into it... the whole thing is 1 hour and a half. I actually think that if I had to waste 97 minutes of my life for something so void, I would probably enjoy more playing with the actual DVD disc for its intriguing prismatic effects.

Some movies are so good that one must see them, some are so bad that one has to see them.. this movie is so empty and unfunny that it really has no point. Or maybe one could create the category of movies that are so unintentionally unfunny that one has to rent them to try watch the first 5-10 minutes to feel what is like to be lobotomized.

Very sad indeed.

One of the reasons why I left Italy was that Italians talk and argue about politics constantly, as if all you had to do was to cast a vote to some mythical "right party" to fix all the problems at once.

Recently I've been reading the Italian news again, between the plain ansa.it and the opinionated beppegrillo.it, the blog of Beppe Grillo, a controversial standup comedian banned by TV for his political jokes and that later on gained some serious creed by predicting one of the biggest corporate scandals in Europe.
Mr. Grillo's blog is one of the most frequented in the World, now being translated in English and Japanese. But I doubt that many people outside Italy actually follow it as only the Italian version gets between 3000 and 4000 comments per post.. too many to make any sense of them, and frankly most of those comments are just random venting.

With Naples drowning in its own trash for almost half a year now, there is definitely a lot to talk about.
It seems that Italy managed to stop funds to one of our few Nobel Prize scientists, Carlo Rubbia, which is famous for nuclear research, and that has been working on using solar energy. The research was first founded in Italy, but then he had to move to Spain... while the Italian politicians are pushing for Nuclear energy (banned by a referendum 20 years ago) and at the same time burning trash with cancer-facilitating incinerators, that for the occasion have been renamed as "thermo-valuers".

At the root of all this there is culture. Italians are not altruistic at all and are relatively ignorant on how things go about in the rest of the World.
One could argue that the United States of America is also a fairly closed country where news feeds are rather limited (see Anna Miller's 2008 talk on TED).
USA however is a country with so much active research and development. It gathers the brightest minds from all over the World and to some extent it can afford to have the general population not being too concerned or too educated about what happens outside (there goes the Iraq war..).

Back to the Italian situation. I believe the country needs people to simply go to school, study a lot, study scientific matters, develop logical skills and also study abroad.
We need more and more people to stick their necks outside and see on a practical level how things work in the rest of the World. You can't improve much if you have no practical experience of what other countries are doing right.
In the short term however, before the average cultural level grows, we need to start making sure that people that end up with political power are not your "average Luigi".
People need to face that for a special job one needs special people.
A person with power to govern should have, first of all, a basic moral integrity. A stable personality, great intelligence and motivation (aside from getting rich and powerful !).
Not everyone needs to be a physics Nobel Prize, but that would be ideal.

Most of all, politicians mental abilities and economic activities should be constantly monitored. There should be periodical reviews in which politicians' goals and achievements are being shown to the greater population (leaving some room for national security, but not too much).

Basically, I want to know that whoever is taking decisions for me is actually more fit than me, if not, then I should be doing his job !
Politics shouldn't be a career, it should be a service. Granted that experience is a key factor in everything, but experience is also an excuse for people to get lazy and to make huge mistakes on the behalf of millions and millions of people.
Being a politicians is a job of great responsibilities, it ought to employ the brightest minds and put them constantly under scrutiny !

..in the meantime, Italian parliament and senate includes convicted criminals and shady individuals.
Get those out first !

I've been coding at home.. with my iMac.. only, using Windows XP 8)

While learning new OSes and new development environments or even new languages (Objective-C !) can be a great thing, my current focus is really mouse about graphics that application interfaces or multi-platform development.
I'm also so much used to Visual Studio (.NET 2005, using VC6 key mapping) and Visual Assist X.

I'm only now managing to get a decent setup and here is a list of things that are helping me:

VMWare Fusion

Fusion can actually display single application windows mixed with the OS X windows, but personally I prefer to leave it in a single window, better if full-screen.. because I can get OS X windows on top of it (basically full-screen means "maximized", though there is no "maximize" in OS X).

One problem I had, is that the cursor stays the Mac one, meaning that I get a black caret on my black background code editor windows 8)
For that I went to change cursor type on the Windows side.

Another key factor is having 3GB of RAM..

A Real Mouse !!

With one button one has to use the Control key to simulate right click. Which is fine, until you try to do Control+LMB 8)

Only today I discovered that Apple's Mighty Mouse (standard for iMac) can actually handle a right click !! Great ! ..not really: it's very easy to accidentally do a right click while trying to do a left one. A very frustrating experience, possibly worse than the unwanted side buttons clicks, the scroll ball/nipple getting stuck and sometimes the inability to click at all. What a crappy mouse !

My solution was to plug a regular 2 buttons + Scroll Wheel mouse.. much better now.

Key and mouse mappings

I set the middle mouse button (clicking on Scroll Wheel) to show "All Windows" (see Expose' video). The "All Windows" view is actually very useful.

Then I swapped Control and Alt (Option on Mac), because on the iMac keyboard Alt takes the place of Control which is too much an important key to try attempt relearn it's position (when I'm not at work !)

...And ?

My smallish "computer desk" is still lacking. Must get rid of it, but trashing old furniture and housing large furniture does require a certain will power if you live in Tokyo 8)

I wish I had two smaller monitors rather than a large 24 inches one. In the office, I like how I can maximize Visual Studio in the left monitor while doing something else in the right monitor.

Still, I'm much happier now. I can use the same computer using applications from both OS X and Windows.
Interoperability is great. Copy and paste, drag and drop works both ways.
Keyboard and mouse are just too important, and while I can tolerate XP not recognizing the Japanese layout (could overwrite a DLL for that, but I prefer the English keyboard layout for code anyway), I definitely needed to remap the Control key and to get a decent mouse.

I'm writing this with an editor on the Mac while the Windows screen saver runs in the background. Putting an x86 processor on the Mac is the best thing Apple could have done for the machine at least for those of us that can't get into serious relationships (with operative systems 8)

woooo !!

Yesterday night I went to ageHa club in Tokyo to listen to Ferry Corsten.
I'm a big fan of Trance music.. it was great.. if only I weren't thinking all the time that I wanted to go back home and code !!

As I pretty much still develop on Windows, I've been using my Vaio laptop at home (not laptop from the company, they promised it once, but then took it back 8( ).
At home I only have a small desk and that's for the iMac. So when I use the Vaio it's on the kotatsu, with it's limited monitor and keyboard... not the best environment for developing (it's important to have a full size keyboard to write code !).

So, yesterday I decided to try installing Visual Studio on the iMac under VMWare Fusion. It works surprisingly well (I have 3GB of RAM). DX9 support doesn't seem to be there, but it's probably a setting or an upgrade. In any case recently for me it's either DirectX 10 (ewww) or software rendering (slooww).

Speaking of software rendering, I started with the idea of implementing a RenderMan state machine and borrow from RenderMan Shading Language. But the tendency is more towards Mental Ray.. which interestingly implements shaders as DLLs compiled in C/C++ (I've heard that Pixar itself does use all sort of languages to write shaders).

Writing shaders in C++ is one of my goals. HLSL/Cg kind of shaders are rather limiting, but for a good reason, those limits make it easier to write compilers that can parallelize operations. Still those limits are becoming unbearable and if shaders are written in C++, they can potentially run at the same level of a 3D engine on the same platform without having to create all those silly contractual interfaces like it happens with HLSL for example.
There will always be a problem with optimizing the interface between engine and modular shaders or any kind of module, but still, not having to worry about heavy state changes, buffering hardware display lists, etc etc, is going to make it easier to solve certain problems in different ways.

I see HLSL/Cg shaders like the (Macromedia) Flash of real-time 3D graphics: an excellent platform that can bring to great results with minimal effort, but a very limiting platform, eventually forcing a whole mindset to down the throat of an entire generation of graphics programmers.

It's important to be nonconformist sometimes !

woooooooo

It's really fucked up to think that to draw a quad one has to has to create vertex buffer resource, an "input layout" and whatnot.

It's lame but I must do it, but it's really lame !

Fack those ill conceived abortions (oxymoron ?).. started with Direct3D 1.0

Everyday I see the design mistakes I make when I write any code.. and then I think that it's the same mistakes that every programmer does.. only that most of us have to deal with the bullshit of few.

Direct3D 10 is bullshit... and so is Vista.. but that's nothing new 8)

Oh and by the way, I managed to work around the nVidia driver crash bug in Vista... all I had to do was to scale down the geometry 8)
I had position values going beyond 160000.0f, I scaled down to 50000.0f and the crash is gone.

Pretty fucked up 8)

UPDATE: It turns out that the problem arises with the size of the object relative to the far plane. I guess it's basically a far plane clipping issue.
In fact I had the driver crash again when pulling closer the far clipping plane on a scene that would work fine before.
So there you have it, now fix that driver, please !

Recently, I felt the pain of hands-on C++ development again.

Because I use SSE2 instructions and because those need to work on at least 16-bytes aligned memory, using standard "new" and "delete" was out of the question.
I proceeded in overriding them globally and wxWidgets started working funny: it sometimes would use my globally defined "new" and "delete", sometimes it would use the standard Microsoft CRT.
I finally found out that I should have enabled global memory allocation using a specific option in wxWidget's "setup.h" file (a configuration file for compilation).

Next, I decided that I needed memory leak detection. But, because I'm overloading the allocation operator I can't use the MS CRT's leak detection.
For the memory manager to keep track of allocations I decided to use a fixed length STL-like vector. This vector (which basically contains an array) is built as a template class that accepts as parameters the type and the maximum size.

The vector was declared as static and this gave me another big problem that took me a while to track down. At startup my poor vector was actually being used as some other static functions from wxWidgets called "new" and "delete" before the vector was actually initialized.

Basically the issue being lack of dependency between statically allocated items (a well known issue in C++).
One way to fix that was to put the array into a function. Every time a "new" was called, it would call a function which would initialize the static array the first time.
This is called "construct on first use". It worked well to command initialization.. but would leave me in the dark on where the destructor would be called.

I eventually built a separate vector class without constructor and destructor 8)

I've also cleaned up a bit things with smart pointers... but it's really a mess with C++. On the other hand, the approach of languages designed form ground-up with automatic garbage collection doesn't seems like a great thing for performance when seeing how those things work when implemented in C++.. lots of work behind the scenes !
But then, maybe one shouldn't use pointers when performance is of essence.. easier said than done !

Friday I got to test a new model that I'll have to use for my current project.
The unoptimized model had over 1M polys (it's a small one 8). I went on to display it with the DX10-based engine and it crashed the (NVidia) driver.
The model is actually composed of many smaller models, so it's not like there is a huge vertex buffer. It also had no textures and I was using a simple common shader.
The only issue I can think of is the range of coordinates which it's relatively very large. It's a laughable supposition, but everything points to that. I'll have to debug to find a workaround.. but it's not easy considering that those driver crashes force me to reboot almost every time ! (Vista stays up, but I can't do 3D anymore).
This is a really bad setback. For the test I eventually displayed the model with my ultra-basic software renderer... no crashes there ;) ..and if there were, I could have fixed them myself.

Another thing that bothers me about DX10 and drivers in general, is how one has to guess performance, because the internals are obscure.
For example, using an NVidia 8800, I noticed that performance is a lot worse when using buffers flagged as "dynamic".
This whole "static" vs "dynamic" thing is apparently part of DX10 and Vista's driver model. Somebody, somewhere, probably decides to put the buffer in system memory (as opposed to GPU memory) under the assumption that the buffer needs to be touched frequently. Only, I may want to change it rarely, and also I was sort of expecting for the buffer to be allocated directly in the GPU memory and only be mirrored in system memory if I ever tried to read from the buffer (which I wouldn't dare to).

So, I have to be really careful and only use the "dynamic" flag for things that change frequently.. and possibly forget about building a flexible system that uploads textures and geometry on-demand.. which is otherwise theoretically very possible with no (not much ?) performance degradation.
As it stands, it seems almost that dynamic buffers are being uploaded per-frame, regardless of the fact that they aren't being modified per-frame.
..this is all speculation of course.. but that's what I really don't like about this: having to spend time trying to guess what those drivers do behind my back.. and hope that different drivers on different card will behave similarly (crashes aside ;).

For this reason I hope that the time will come when game companies can write complete graphics pipelines again. Either in-house or licensing code, but staying away from closed-source drivers, so that one won't have to debug and profile in the dark.

Some are worried that they couldn't possibly do much better than what card manufacturers already do with those drivers.. I think that there is plenty to improve by just getting rid of those fat drivers that have been plaguing PCs ever since 3D cards came out.

ole'