Dewaniya Net

Remember the project I was working on? Well it’s not quite there yet, but I’ve decided to reveal it anyway to spark people’s interests. It’s called, “Dewaniya Net” and in essence, the idea is about linking people’s digital content to create a unified Kuwaiti network. You can read more about the many advantages of doing so, how the name came about, as well as how to build your own server here.

My friend and I are still debating on which protocols we ought to use; we thought Bittorrent would be the obvious way to go but then realized its complications when it came to generating numerous individual torrent files.

I’m especially interested in what Bojacob and Marzouq have to say about this. Other geeks are welcome to discuss ofcourse :P

25 thoughts on “Dewaniya Net

  1. Man this is a very gutful move I salute you for it. It’s a big one and I’m sure you’re fully ready for the challenges ahead. Your solution could get rid of the BitTorrent blocking problem. I am not sure however, but does it really bring down the cost a lot? I mean is the biggest problem with torrents is that they are mostly from outside Kuwait?

    Also, I hope you get your self covered as you are liable for what content is distributed, and I can only guess loads of pirated materials.

    In the sample File listing, I noticed the download option is a zipped file of all directory. Would that be a waste of bandwidth if I only want one album say. Perhaps a smarter way for the future would be for me to select which files I want, then compress it on the fly and download it. Ofcourse, added overhead on the server processing time.

    For the storage, did you consider using SAN, perhaps i would be easier for expansion and more powerful? I’m sure you know better why you made that choice, but I’d like to know why.

    And last, beside Good luck again, hope MOC and ISPs don’t trace your routes and block your own connection :(

  2. The idea is really good, and I like how you guys are trying to get it all distributed..

    I think a private tracker is the best way so you can selectively add people..

    one issue we have in Kuwait is bandwidth… it will eat up your bandwidth..

  3. Overall I think you guys are aiming for people who are more technical who know how execute terminal commands..

    80% of Kuwait won’t be able to do this..

    Probably a few of us can do this. .

    The bandwidth problems we have in Kuwait is from the ISP not just through MOC… so its something to think about…

    different ISPs have different ways of routing internal traffic.. so thats is something to think about..

    But still a great project worth working on..

  4. I have been thinking about the idea since MBH told me about it. Like the others have said, there are two issues .. bandwidth and content.

    A simple (from technical perspective) is a central database in Kuwait connected to a 100mb port at one of the ISPs; and the Diwaniya nodes can upload/download content. Every node connected would pay a subscription charge to maintain the central database.

    Me and Marzouq have a little over 10TB shared between us .. more than happy to provide hardware/content.

  5. Bashar, costs go down if traffic circulates within Kuwait and not from abroad. The content would have to be encrypted as these are personal/private files that are shared between friends. The file listings you saw was just that, a list. We haven’t agreed on what protocols to use for downloading and distributing yet. I don’t think we can afford SANS :/

    Linus, great.

    Ms. D, um.. not quite…

    Amethyst, Thank you.

    forzaq8, Thank you for your suggestion. We’ll be look into it.

    Sushi, Thanks.

    Marzouq, it’s not practical to generate thousands of torrent files and what’s to stop bittorrent clients from connecting to external sources?

    There are other methods of joining the network. You don’t necessarily need to build a server.

    K.TheKuwaiti, the node admins can discuss what content they want to obtain and serve amongst themselves. Please elaborate on the “subscription charge” idea since I don’t see why anyone would want pay to connect to a node that may not even be reliable to begin with.

  6. The subscription charge would go towards creating a reliable node with high bandwidth capacity.

    In terms of sharing content, the simplest way is to create a Kuwait-only tracker. As the biggest benefactors would be the local ISPs (from conserved bandwidth) .. they could in turn allow for higher speeds within Kuwait (or host the tracker itself).

  7. I’d love to help you there 3baid but my knowledge for this stuff (as probably you know already) is zilch. However, if the project worked then I’d be happy to contribute by sharing some files with other people.

    Good luck :)

  8. And hosting on a home-dsl connection would be impossible. So the best way is to get a dedicated server overseas and configure a private Kuwaiti tracker. Node-Admins can make requests for content thats not currently being served.

  9. The issue with providing content is the upload bandwidth.. The majority of nodes would have a maximum uploads capacity of 60kbps each.

    Seeing how a dedicated server in Europe could provide roughly 100x that amount .. its hard to convince people to share within Kuwait when speeds elsewhere are faster. Unless people come together once a month to swap harddrives.

  10. Marzouq, we’re not aiming at technical people at all.
    We’re documenting our project because the technical information involved could be beneficial to anyone over the world.

    As for our aim, it’s to create one working node and then we can bundle everything as a Linux distribution by itself, and provide configuration to it through a web interface.

    K, I know what you mean but that would void the goal of this project: keeping the bandwidth local.

    Our dewaniya have a lot of online gamers, so during gaming sessions, ping times must be maintained low, so we’re going to make schedules for upload speeds depending on such times.
    As for the amount of upload, you’re right, it’s not much, but the more people have the file, the more peers you get (likely only for popular files).

    As for renting a space in an ISP’s datacenter in Kuwait, that’s just way too expensive and knowing the kind of ISPs we have, they’d pull the plug on our boxes without prior notice, anytime they want, and all our content goes down the toilet.

    Maybe we can avoid ISPs in general by having everyone stick a wireless antenna on their roof and create one giant wireless network! (yay for virus out breaks!)
    SOHO wireless routers are limited to 255 clients only anyway :/

    Comcast in the US now provide 50Mbps/6Mbps connections, and Velchip in Indonesia is providing 224Mbps over power lines, and here in Kuwait, one of the richest countries, we’re stuck with 256 & 512kbps download/upload streams…

  11. I forgot to mention that the tracker idea is not feasible. It was our intention to use it at first, but then canceled due to the fact that each “node” will have over 30,000 files and since we’re aiming at cheap machines & storage, they won’t be able to handle that amount of files.

    Being a tracker means having to create many many connections, which would break the server, the modem or the router.

    We’re looking at programs similar to DC++ at the time being. If it has a command line and a web, we want it!
    queues, bandwidth control & schedules are also required.
    If it doesn’t have a web interface, it’s OK, we’ll make one!

    Our node is very simple and cost us 60KD (minus the case & disks). We have no graphical interface & running on 1GB, 105MB of it are currently used!!! (LINUX BABY! YEAH!)

  12. DC++ (or variations) would actually work quite well in this situation .. limit the ip ranges to only those within Kuwait. I am putting together a DewaniyaPC next week and adding it to the nodes. The best ISP locally would be Zajil, their upload speeds are better than most.

  13. K,
    Part of building the network is having “trusted sources” — these will be the main nodes and they will have the domain names of other nodes. All other nodes update their lists from these trusted sources.

    We *could* limit access to nodes using Kuwaiti ISP IP blocks, but that would make it more difficult to add newer blocks in the future, as it would require manually configuring the new IPs on Linux iptables.

    We’ll see what Zajel offers, but we have to keep in mind, that the Internet links which house DewaniyaNet nodes are also used by home owners and their first interest would be their own needs, before sharing with people.

  14. It will depend on the owner of the box. You know our dudes, they abuse it from 5 Pm till 3 AM…
    As for my box at home, usage is usually from 2 PM till 11 PM, though I have Azureus uploading all the time (sharing is caring!)

    Maybe at an advanced stage of the project, we could create “node caches” .. nodes with fast links could host files with high demand…

    I should have sometime this weekend, so I’ll start testing clients on the virtual machine.

  15. If this picks up .. I would most likely dedicate a 1mb connection just for the box, my main link is congested as it is.

    MBH, If you need another node, I should have the pc running next week for running remote tests .. just waiting for some drives to come in.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s