You can run this locally on a PC so what are the advantages of having it on flarum?

    It's consuming too much RAM.

    1. Memory Requirements
      Runs on most modern computers. Unless your computer is very very old, it should work.
      According to a llama.cpp discussion thread, here are the memory requirements:

    7B => 4 GB
    13B => 8 GB
    30B => 16 GB
    65B => 32 GB

    Not goot for any server

      Wellwisher
      I think this would be incredibly FUN if you have a AI User that you can ask any questions who is directly answering in the Thread... 👍️

      BilgehanArslan
      I think this refers to the Harddisk Usage, not RAM...
      For Example, you can install it on a Raspberry Pi

      Oshvam
      If I am not mistaken, they are working on an Plugin implementing the original ChatGPT, not the Open Source DALAI Version...

            I don't think any resource requirement is a deal breaker for any feature. For some people it will be worth it. It's always balance a of abilities/quality and cost between cloud options and self-hosted options.

            Even if running the model requires a large amount of resources and budget is tight, a single instance could be shared between multiple customers by sharing a server and splitting the costs, or depending on the use case it could run on a schedule on a powerful virtual machine that's only rented for a few minutes or hours per day.

            What I wonder is whether that dalai application brings any benefit. It just seems to be a UI built on top of other libraries. If integrating with a Flarum extension, it might not make any sense to pull the dalai source code but rather directly hit the underlying libraries that provide API access for applications. The nice UI on top won't make it easier to implement into an extension apart from serving as implementation example perhaps.