How Intel's open source Data Plane Development Kit enables high-performance Linux networking - TechRepublic: "To utilize processors more efficiently in Linux, developers bypass the Linux kernel and dedicate CPU cores to network processing. The CloudFlare example used Linux kernel tools to pin processes to a set of cores and actively managed memory queues. It took a bit of programming discipline to send unprocessed packets between two systems. Add any packet processing, such as IPS, and the performance should drop significantly.
DPDK bypasses the Linux kernel and manages the NIC and CPU assignment directly. On a single Intel processor, it's possible to achieve throughput of up to 80Mpps or roughly 46Gbps using the smallest Ethernet Frames. I want to make sure I caveat the above comment and note that it was an Intel processor. DPDK is an open source project and currently has both Intel and ARM distributions."
'via Blog this'
Showing posts with label networking. Show all posts
Showing posts with label networking. Show all posts
Wednesday, 24 August 2016
Sunday, 10 July 2016
Qcon: Much Faster Networking
Qcon: Much Faster Networking
David Riddoch talks about technologies that make very high performance networking possible on commodity servers and networks, with a special focus on kernel bypass technologies including sockets acceleration and NFV. These techniques give user-space applications direct access to the network adapter hardware, making possible sub-microsecond latencies and millions of messages per second per thread
David Riddoch talks about technologies that make very high performance networking possible on commodity servers and networks, with a special focus on kernel bypass technologies including sockets acceleration and NFV. These techniques give user-space applications direct access to the network adapter hardware, making possible sub-microsecond latencies and millions of messages per second per thread
Thursday, 31 March 2016
Linux IOVisor versus Data Plane Development Kit
Data Plane Development Kit - Wikipedia, the free encyclopedia: "The Data Plane Development Kit (DPDK) is a set of data plane libraries and network interface controller drivers for fast packet processing. The DPDK provides a programming framework for Intel x86 processors and enables faster development of high speed data packet networking applications.[1][2] It scales from Intel Atom processors to Intel Xeon processors and support for other processor architectures like IBM POWER8 are under progress.[3] It is provided and supported under the open source[4] BSD license."
http://www.linuxfoundation.org/news-media/announcements/2015/08/test
“IO Visor will work closely with the Linux kernel community to advance universal IO extensibility for Linux. This collaboration is critically important as virtualization is putting more demands on flexibility, performance and security,” said Jim Zemlin, executive director, The Linux Foundation. “Open source software and collaborative development are the ingredients for addressing massive change in any industry. IO Visor will provide the essential framework for this work on Linux virtualization and networking.”
"Advancing IO and network virtualization in the Linux stack can be an enabler of agility and elasticity, which are key requirements for cloud deployments and applications. IO Visor Project’s mission to bring universal IO extensibility to the Linux kernel will accelerate innovation of virtual network functions in SDN and NFV deployments,” said Rohit Mehra, Vice President of Network Infrastructure, IDC. “The ability to create, load and unload in-kernel functions will enable developers in many upstream and downstream open source projects. What’s more, as an initiative under the auspices of the Linux Foundation, the IO Visor Project has the potential for credibility and momentum to benefit the diverse community of vendors and service providers, and ultimately enterprise IT.”
Monday, 28 March 2016
What happens if you write a TCP stack in Python? - Julia Evans
What happens if you write a TCP stack in Python? - Julia Evans:
"During Hacker School, I wanted to understand networking better, and I decided to write a miniature TCP stack as part of that. I was much more comfortable with Python than C and I'd recently discovered the scapy networking library which made sending packets really easy. So I started writing teeceepee!
The basic idea was:
I didn't care much about proper error handling or anything; I just wanted to get one webpage and declare victory :)"
Traceroute in 15 lines of code using Scapy
'via Blog this'
"During Hacker School, I wanted to understand networking better, and I decided to write a miniature TCP stack as part of that. I was much more comfortable with Python than C and I'd recently discovered the scapy networking library which made sending packets really easy. So I started writing teeceepee!
The basic idea was:
- open a raw network socket that lets me send TCP packets
- send a HTTP request to GET google.com
- get and parse a response
- celebrate!
I didn't care much about proper error handling or anything; I just wanted to get one webpage and declare victory :)"
Traceroute in 15 lines of code using Scapy
'via Blog this'
Why you should understand (a little) about TCP - Julia Evans
Why you should understand (a little) about TCP - Julia Evans:
"Delayed ACKs & TCP_NODELAY
Ruby's Net::HTTP splits POST requests across two TCP packets - one for the headers, and another for the body. curl, by contrast, combines the two if they'll fit in a single packet. To make things worse, Net::HTTP doesn't set TCP_NODELAY on the TCP socket it opens, so it waits for acknowledgement of the first packet before sending the second. This behaviour is a consequence of Nagle's algorithm. Moving to the other end of the connection, HAProxy has to choose how to acknowledge those two packets. In version 1.4.18 (the one we were using), it opted to use TCP delayed acknowledgement. Delayed acknowledgement interacts badly with Nagle's algorithm, and causes the request to pause until the server reaches its delayed acknowledgement timeout.."
'via Blog this'
"Delayed ACKs & TCP_NODELAY
Ruby's Net::HTTP splits POST requests across two TCP packets - one for the headers, and another for the body. curl, by contrast, combines the two if they'll fit in a single packet. To make things worse, Net::HTTP doesn't set TCP_NODELAY on the TCP socket it opens, so it waits for acknowledgement of the first packet before sending the second. This behaviour is a consequence of Nagle's algorithm. Moving to the other end of the connection, HAProxy has to choose how to acknowledge those two packets. In version 1.4.18 (the one we were using), it opted to use TCP delayed acknowledgement. Delayed acknowledgement interacts badly with Nagle's algorithm, and causes the request to pause until the server reaches its delayed acknowledgement timeout.."
'via Blog this'
High Performance Browser Networking
High Performance Browser Networking:
Our goal is to cover what every developer should know about the network: what protocols are being used and their inherent limitations, how to best optimize your applications for the underlying network, and what networking capabilities the browser offers and when to use them.
In the process, we will look at the internals of TCP, UDP, and TLS protocols, and how to optimize our applications and infrastructure for each one. Then we’ll take a deep dive into how the wireless and mobile networks work under the hood—this radio thing, it’s very different—and discuss its implications for how we design and architect our applications. Finally, we will dissect how the HTTP protocol works under the hood and investigate the many new and exciting networking capabilities in the browser:
- Upcoming HTTP/2 improvements
- New XHR features and capabilities
- Data streaming with Server-Sent Events
- Bidirectional communication with WebSocket
- Peer-to-peer video and audio communication with WebRTC
- Peer-to-peer data exchange with DataChannel
Understanding how the individual bits are delivered, and the properties of each transport and protocol in use are essential knowledge for delivering high-performance applications. After all, if our applications are blocked waiting on the network, then no amount of rendering, JavaScript, or any other form of optimization will help! Our goal is to eliminate this wait time by getting the best possible performance from the network.
High-Performance Browser Networking will be of interest to anyone interested in optimizing the delivery and performance of her applications, and more generally, curious minds that are not satisfied with a simple checklist but want to know how the browser and the underlying protocols actually work under the hood. The "how" and the "why" go hand in hand: we’ll cover practical advice about configuration and architecture, and we’ll also explore the trade-offs and the underlying reasons for each optimization.
Friday, 18 March 2016
SDN OpenDaylight Project
OpenDaylight Project - Wikipedia, the free encyclopedia:
The OpenDaylight Project is a collaborative open source project hosted by The Linux Foundation. The goal of the project is to accelerate the adoption of software-defined networking (SDN) and create a solid foundation for Network Functions Virtualization (NFV). The software is written in Java.
The OpenDaylight Project is a collaborative open source project hosted by The Linux Foundation. The goal of the project is to accelerate the adoption of software-defined networking (SDN) and create a solid foundation for Network Functions Virtualization (NFV). The software is written in Java.
The Path to SDN
http://go.bigswitch.com/rs/974-WXR-561/images/The%20Path%20To%20SDN%20Final.pdf
Networking has seen more innovation in the past two years than in the past 20 years. This innovation is coming from the perfect storm of:
• ‘Software-Defined Networking,’ a new architecture in networking
• ‘White box’ and ‘brite box’ switching
• The popularization of advanced datacenter designs pioneered by Google, Facebook, Amazon, and other hyperscale operators
A Look Back at How It All Started
Bruce Davie,: Is SDN Necessary?
Is SDN Necessary?:
SDN != Network Virtualization SDN control control Network Virtualization != Virtualization layer control SDN alone doesn’t abstract away details of physical network SDN not required for network virtualization, but useful
'via Blog this'
SDN != Network Virtualization SDN control control Network Virtualization != Virtualization layer control SDN alone doesn’t abstract away details of physical network SDN not required for network virtualization, but useful
'via Blog this'
tcpdump is amazing - Julia Evans
tcpdump is amazing - Julia Evans:
Let's suppose you have some slow HTTP requests happening on your machine, and you want to get a distribution of how slow they are. You could add some monitoring somewhere inside your program. Or! You could use tcpdump. Here's how that works!
The secret here is that we can use tcpdump to record network traffic, and then use a tool that we're less scared of (Wireshark) to analyze it on our laptop after.
Let's suppose you have some slow HTTP requests happening on your machine, and you want to get a distribution of how slow they are. You could add some monitoring somewhere inside your program. Or! You could use tcpdump. Here's how that works!
- Use tcpdump to record network traffic on the machine for 10 minutes
- analyze the recording with Wireshark
- be a wizard
The secret here is that we can use tcpdump to record network traffic, and then use a tool that we're less scared of (Wireshark) to analyze it on our laptop after.
Subscribe to:
Posts (Atom)