Week of the ReRender
Posted by Marius Oberholster on Thursday, February 22, 2018 Under: WIP
Hey all!
This week's vlog!
GOD is so wonderful!!
- Re-Rendering a lot
Like I mentioned in the video, after seeing the issues with the upscaling (after noise reduction and even doing them together), I re-rendered the shot like GOD wanted in the first place. HE did give me leave to keep it as-is and just noise reduce and later upscale, but you really lose out on quality... It's like the smear between DVD and VHS - it's really obvious - not indetail, but in overall finish.
Which leads me to my next reason for soooooo many re-renderings - 3D
- New 3D repair method
The 3D method I use is pretty much parallax scrolling (or shifting in my case). This means that while I only have 10 layers, the visual effect is there. Previously, I attempted repairing everything on the background with only the rest moving, but this fails horribly passed a certain point, so GOD gave me a new method of cleaning up the information that is missing - depth map manipulation.
Basically, like I mentioned in the video, this means that I get to repair the video based on the actual tearing difference, not the mask that is produced using the depth itself. Of course this does require some editing to get it right and get that black and white stark mask, but it works and I am very very thankful for it. The greatest part for me is how GOD led me into it so accidentally. I didn't even plan on doing it this way, it just worked. HE is so awesome!! That's why I said I am not entirely sure about how or why it works, just that it does, haha. I think HE does it this way on purpose so I would look stupid if I try to steal HIS glory in any way, haha. It sounds mean, but it's HIS work, I can't take credit for it - if I do, I deserve to look stupid, haha.
Anyway, because this new repair method worked sooo well and gave me a very crisp result, it meant that things had to be re-rendered. By this time, I had already rendered and added the 3D part of the new scene. So, I had to render it again with the new method. Of course, a few additional improvements came, such as the filling in of the background. A few methods were tested, so of course, multiple renderings followed until the background was simply filled in after being shifted. Then I rendered it directly to anaglyph, which was a mistake and that was another large part of the day wasted to a another re-rendering, haha.
Now that the new method is so much cleaner, I also wanted to redo another scene - the frogs in the floor. There the new method showed it's weakness - slow reveals! Eeeeeeeck! It was awful! So I am using the old method with the part where Pharaoh's assistant reveals and the new method where the floor opens. I don't recall having re-rendered any other scenes, but these were plenty already, hahaha.
And of course, here is the link to Gimpel3D. I mentioned I would share it in the video. All recommendations and links are used entirely at your own risk.
http://www.gimpel3d.com/
- Super resolution methods
Nvidia has a new Super Resolution tool they want to incorporate into their game tools. Since it's in Beta and I don't have access to testing for game dev software, I don't get to play with it until I am allowed to download it (hoping it does sequences too, because games can have a tremendous amount of textures that need to be upscaled). Anyway, I would luv to give that one a try when it comes it!
https://developer.nvidia.com/gwmt
Just wanna be clear, none of these people sponsor me in any way, so don't think of any of these as a paid review kinda thing. I'll definitely let you know if that's the case.
One I mentioned in the video is called Video Enhancer - which is made by infognition. They have done a lot of advertising on their website with before and after results, but really, I don't get anywhere near those results... I am severely disappointed by that, but I totally get that their method is very footage dependent. Because I don't know what footage really produces the best kind of result (except slow panning), I can't use it.
After trying it and uninstalling it, I suggested that they look at incorporating other SR methods as well, to compensate for this shortcoming (that not all footage works well with it), but they just kinda blew me off with 2D won't work, because it's synthetic. I think they missunderstood what I meant by 2D style, but whatever. Point is it doesn't work for Exodus. If you wanna give it a shot, they have a free trial you can throw some footage at here:
http://www.infognition.com/videoenhancer/
Another I am truly amazed by is MDSP Super Resolution. Like VE (Video Enhancer), it also works with sequential frames, but does some other things too and oh wow!!! Unfortunately it has two drawbacks:
- Non-commercial
- Requires an older version of MatLab and MatLab isn't in the budget (I wouldn't even know how to use it anyway, hahaha).
If you have the particular matlab required and you wanna give their stuff a go, here:
https://users.soe.ucsc.edu/~milanfar/software/superresolution.html
The one I mentioned in the video that I threw the 320x240 image at, was called DeepAPI. It has a few stunning features that include Super resolution and inpainting (a lot like what I mentioned in the video, but not quite what I have in mind). It was the one that gave that news-print kinda effect, but still did a fantastic job!
You can try it for yourself - they do offer a service to upscale stuff for you, but you pay per set of a 100 or a 1000. I don't recall, but it is a lot cheaper than a render farm for full rendering, hahaha.
https://deepai.org/machine-learning-model/torch-srgan
Of course with our current data limits, I can't afford it, as cheap as it is.
We cannot leave Google out of the equation with GoogleBrain. They released the results of a study on faces and some rooms:
https://syncedreview.com/2017/03/31/pixel-recursive-super-resolution/
https://arstechnica.com/information-technology/2017/02/google-brain-super-resolution-zoom-enhance/
It is quite stunning and I think it would do a marvelous job on a 2x upscale! They already use such a program on Android devices for G+ and it's called RaisR. Very cool!
And last, but simply not least, YUVSoft's Super Resolution program. This one I haven't tried, but the results I have seen have been pretty impressive. The sequential SR method really help a lot with noise reduction. I do believe, based on their 3D suite, that they would make use of multiple method to produce a proper SR result. I have to say though, the ones that have a library of images or stunning deep learning, really do produce stunning results!!!
http://www.yuvsoft.com/2d-technologies/super-resolution/
Now only the inpainting needs to achieve the same results. I do believe that inpainting based on sequential frames can also be extremely effective. I've seen it in the rotoscoping mode for YUVSoft's 2D-3D conversion. It is definitely one for the record books! haha.
So yeah, that is it for this week. I think I replaced the render from last week with the better rendered one and upscaled it, so I don't have a link for that. Sorry... To make up for it, I made this of the difference between the two frames. The actual two upscaled frames (the one that's cleanly rendered and the one that has been denoised and upscaled):
- Friendly reminder
Of course there will always be people with other ideas, haha! This week, Raul also put a proposal for some NPR tools for Blender and he really did a good job. Go give him a thumbs up too! I really believe his proposal needs to be added as well:
https://blender.community/c/rightclickselect/p2bbbc/npr-nodes-proposal-for-cycles-2-8x
Mine and Wasili's Link here:
https://blender.community/c/rightclickselect/11bbbc/npr-in-2-8
Go give those some up votes, share them, spread the word, etc.! :D
And that is it for this week! Very short and to the point, but really interesting nonetheless! hahaha. Thank you for reading, watching, liking, commenting and subscribing! It really means a lot!
Know JESUS yet?
To find out more, you can check out this link:
It gives you access to free Bible translations, free teaching videos and more. Remember, this is a decision with eternal consequence, choose life, choose JESUS. :D
Have a great one!!!
Thank YOU!!!!!!
In : WIP
Tags: god jesus holy spirit blender anime exodus 7-12 kjv progress wip 3d repair tear shaders update release moses pharaoh river speech
Hey all! I've been doing CG work full-time since 2011. GOD has been with me, supported me and taught me all the way. HE tells us to do all things as unto the LORD and that is my goal. I do whatever I believe the HOLY SPIRIT is leading me to do and it's not always easy, but it's always worth it to be obedient!
JESUS is awesome!! If you don't know HIM yet, follow the link above!
