- 0 Posts
- 15 Comments
1·1 year agoIf you are using CPU only, you need to look at very small models or the 2-bit quants.
Everything will be extremely slow otherwise:
GPU:
Loaded Power: 465W
Speed: 18.5 tokens/second
CPU: Loaded Power: 115W
Speed: 1.60 tokens/second
GPUs are at least 3 times faster for the same power draw.
I don’t know what GPU you’ve got, but Lexi V2 is the best “small model” I’ve seen with emotions, that I can just cite from the top of my head.
It tends to skew male and can be a little dark at times, but it’s more complex than expected for the size (8B feels like 48-70B).
Do Q8_0 if you’ve got the VRAM, Q5_KL for speed, IQ4_XS if you’ve got a potato.
My answer to this is yes.
I’m an AI Developer and my only option was to self host because I didn’t want my training data leaking out onto the web and by extension, China and the rest (I trained on my own data, writing, and notes, along with Wikipedia).
Self-hosting gives you complete freedom but also as one other user cautioned, don’t fall down the well/rabbit hole.
Use an executable like LM Studio, and then an off the shelf pre-trained model from Huggingface.
VRAM × 0.8 for max size.
Experiment until you find one you like.
Put four pots over the squares over the ground.
Shoot the dragon head statues, the pedestals raise.
The pedestals make stone grinding sounds and…
Only one pedestal has raised, the pots have caused the animation to bug out and the game engine to assume that the pedestal is in the final position on the floor.
The floor position has the lever locked.
The game developer never anticipated what a massive idiot I was
I’m so mad that my open source software made by a single developer in his free time who loves coding and helping other people is slightly worse in quality than a multi-million dollar dedicated project.
Exports into Maya-3D.Blend
0·2 years agoshaves the sphere down with a sculptor’s knife
There. 3.1416. Not perfectly round but it’ll bake in the oven just fine.
That seems to require a level of foresight and planning that most corporations don’t have. That’s almost like a blueprint for failure when some middle manager changes the scope of a project with a hard coded time limit, IMO.
Anyone interested in not-agile development? Maybe we can call it “Ship it when it’s ready” lol
Ancient developer here / not really a coder, but what the hell is “Agile” software development?
Is it some kind of pseudonym for pushing buggy, untested code to a production server or something?
Like a speed run category for software development?
The answer was to disable FreeSync. FreeSync was causing the stuttering with SAM turned on.
Haha, now show us a map of the size of different stars compared to Sol (ours, in the local solar system).
Sometimes the small stars and communites are the most important. ;)
I just need AV2 to not blow chunks on VR/Streaming.
AV1 is currently hamstrung by decode compute costs, I’m sending 6K video @ 120 FPS and at 200 Mbits (25 MB/s) it looks like pea soup.
At 400-500 Mbps (62.5 MB/s), you have twice as much network overhead but somehow H.264 still decodes faster even though it’s twice as much raw data per second.
For straight up video file compression, AV1 seems to be absolutely unmatched if you use “peak framerate” (say 120) and then VFQ / variable quality set to like 35.
I was able to convert a 900 MB (!!) H264 video of VR gameplay (6K across, 3K/eye!) into 96 MB, and slide right under Discord’s idiotic 100 MB limit.