工欲善其事，必先利其器，没有好的工具，怎么能高效的开发出高质量的代码呢？本文为 ASP.NET 开发者介绍一些高效实用的工具，包括 SQL 管理，VS插件，内存管理，诊断工具等，涉及开发过程的各个环节，让开发效率翻倍。当然，吾乐吧软件站认为.NET神器第一名的绝对是ReSharper，有了这个，基本上其他工具都可以忽略！
AutoMapper: 自动生成对象到对象的映射代码，比如，能够生成从实体对象映射到域对象，而不是手动编写映射代码。Object to object mapping. Like, the tool can be used to map entity objects to domain objects instead of writing manual mapping code.
WireShark: It is a network protocol analyzer for Unix and Windows. It can capture traffic at TCP level.
HTTP Monitor: enables the developer to view all the HTTP traffic between your computer and the Internet. This includes the request data (such as HTTP headers and form GET and POST data) and the response data (including the HTTP headers and body).
// writing on a text file
#include <fstream>
using namespace std;
int main()
{
ofstream examplefile("example.txt");
if (examplefile.is_open())
{
examplefile << "This is a line.\n";
examplefile << "This is another line.\n";
examplefile.close();
}
return 0;
}
从文件中读入数据也可以用与 cin的使用同样的方法：
// reading a text file
#include <iostream>
#include <fstream>
#include <cstdlib>
using namespace std;
int main ()
{
char buffer[256];
ifstream examplefile("example.txt");
if (! examplefile.is_open())
{
cout << "Error opening file"; exit (1);
}
while (!examplefile.eof())
{
examplefile.getline(buffer,100);
cout<<buffer<< endl;
}
return 0;
}
//This is a line.
//This is another line.
read() 从文件中读取 num 个字符到 buf 指向的缓存中，如果在还未读入 num 个字符时就到了文件尾，可以用成员函数 int gcount();来取得实际读取的字符数；而 write() 从buf 指向的缓存写 num 个字符到文件中，值得注意的是缓存的类型是 unsigned char *，有时可能需要类型转换。
例：
unsigned char str1[]="I Love You";
int n[5];
ifstream in("xxx.xxx");
ofstream out("yyy.yyy");
out.write(str1,strlen(str1));//把字符串str1全部写到yyy.yyy中
in.read((unsigned char*)n,sizeof(n));//从xxx.xxx中读取指定个整数，注意类型转换
in.close();out.close();
One of my favorite things about Python is that users get the benefit of observing the R community and then emulating the best parts of it. I’m a big believer that a language is only as helpful as its libraries and tools.
This post is about pandasql, a Python package we (Yhat) wrote that emulates the R package sqldf. It’s a small but mighty library comprised of just 358 lines of code. The idea of pandasql is to make Python speak SQL. For those of you who come from a SQL-first background or still “think in SQL”, pandasql is a nice way to take advantage of the strengths of both languages.
In this introduction, we’ll show you to get up and running with pandasql inside of Rodeo, the integrated development environment (IDE) we built for data exploration and analysis. Rodeo is an open source and completely free tool. If you’re an R user, its a comparable tool with a similar feel to RStudio. As of today, Rodeo can only run Python code, but last week we added syntax highlighting for a bunch of other languages to the editor (markdown, JSON, julia, SQL, markdown). As you may have read or guessed, we’ve got big plans for Rodeo, including adding SQL support so that you can run your SQL queries right inside of Rodeo, even without our handy little pandasql. More on that in the next week or two!
ps If you download Rodeo and encounter a problem or simply have a question, we monitor our discourse forum 24/7 (okay, almost).
A bit of background, if you’re curious
Behind the scenes, pandasql uses the pandas.io.sql module to transfer data between DataFrame and SQLite databases. Operations are performed in SQL, the results returned, and the database is then torn down. The library makes heavy use of pandaswrite_frame and frame_query, two functions which let you read and write to/from pandas and (most) any SQL database.
Install pandasql
Install pandasql using the package manager pane in Rodeo. Simply search for pandasql and click Install Package.
You can also run ! pip install pandasql from the text editor if you prefer to install that way.
Check out the datasets
pandasql has two built-in datasets which we’ll use for the examples below.
meat: Dataset from the U.S. Dept. of Agriculture containing metrics on livestock, dairy, and poultry outlook and production
births: Dataset from the United Nations Statistics Division containing demographic statistics on live births by month
Run the following code to check out the data sets.
#Checking out meat and birth data
from pandasql import sqldf
from pandasql import load_meat, load_births
meat = load_meat()
births = load_births()
#You can inspect the dataframes directly if you're using Rodeo
#These print statements are here just in case you want to check out your data in the editor, too
print meat.head()
print births.head()
Inside Rodeo, you really don’t even need the print.variable.head() statements, since you can actually just examine the dataframes directly.
An odd graph
# Let's make a graph to visualize the data
# Bet you haven't had a title quite like this before
import matplotlib.pyplot as plt
from pandasql import *
import pandas as pd
pysqldf = lambda q: sqldf(q, globals())
q = """
SELECT
m.date
, m.beef
, b.births
FROM
meat m
LEFT JOIN
births b
ON m.date = b.date
WHERE
m.date > '1974-12-31';
"""
meat = load_meat()
births = load_births()
df = pysqldf(q)
df.births = df.births.fillna(method='backfill')
fig = plt.figure()
ax1 = fig.add_subplot(111)
ax1.plot(pd.rolling_mean(df['beef'], 12), color='b')
ax1.set_xlabel('months since 1975')
ax1.set_ylabel('cattle slaughtered', color='b')
ax2 = ax1.twinx()
ax2.plot(pd.rolling_mean(df['births'], 12), color='r')
ax2.set_ylabel('babies born', color='r')
plt.title("Beef Consumption and the Birth Rate")
plt.show()
Notice that the plot appears both in the console and the plot tab (bottom right tab).
Tip: You can “pop out” your plot by clicking the arrows at the top of the pane. This is handy if you’re working on multiple monitors and want to dedicate one just to your data visualzations.
Usage
To keep this post concise and easy to read, we’ve just given the code snippets and a few lines of results for most of the queries below.
If you’re following along in Rodeo, a few tips as you’re getting started:
Run Script will indeed run everything you have written in the text editor
You can highlight a code chunk and run it by clicking Run Line or pressing Command + Enter
You can resize the panes (when I’m not making plots I shrink down the bottom right pane)
Basics
Write some SQL and execute it against your pandasDataFrame by substituting DataFrames for tables.
pandasql creates a DB, schema and all, loads your data, and runs your SQL.
Aggregation
pandasql supports aggregation. You can use aliased column names or column numbers in your group byclause.
# births per year
q = """
SELECT
strftime("%Y", date)
, SUM(births)
FROM births
GROUP BY 1
ORDER BY 1;
"""
print sqldf(q, locals())
# strftime("%Y", date) SUM(births)
# 0 1975 3136965
# 1 1976 6304156
# 2 1979 3333279
# 3 1982 3612258
locals() vs. globals()
pandasql needs to have access to other variables in your session/environment. You can pass locals() to pandasql when executing a SQL statement, but if you’re running a lot of queries that might be a pain. To avoid passing locals all the time, you can add this helper function to your script to set globals() like so:
# joining meats + births on date
q = """
SELECT
m.date
, b.births
, m.beef
FROM
meat m
INNER JOIN
births b
on m.date = b.date
ORDER BY
m.date
LIMIT 100;
"""
joined = pysqldf(q)
print joined.head()
#date births beef
#0 1975-01-01 00:00:00.000000 265775 2106.0
#1 1975-02-01 00:00:00.000000 241045 1845.0
#2 1975-03-01 00:00:00.000000 268849 1891.0
WHERE conditions
Here’s a WHERE clause.
q = """
SELECT
date
, beef
, veal
, pork
, lamb_and_mutton
FROM
meat
WHERE
lamb_and_mutton >= veal
ORDER BY date DESC
LIMIT 10;
"""
print pysqldf(q)
# date beef veal pork lamb_and_mutton
# 0 2012-11-01 00:00:00 2206.6 10.1 2078.7 12.4
# 1 2012-10-01 00:00:00 2343.7 10.3 2210.4 14.2
# 2 2012-09-01 00:00:00 2016.0 8.8 1911.0 12.5
# 3 2012-08-01 00:00:00 2367.5 10.1 1997.9 14.2
It’s just SQL
Since pandasql is powered by SQLite3, you can do most anything you can do in SQL. Here are some examples using common SQL features such as subqueries, order by, functions, and unions.
#################################################
# SQL FUNCTIONS
# e.g. `RANDOM()`
#################################################
q = """SELECT
*
FROM
meat
ORDER BY RANDOM()
LIMIT 10;"""
print pysqldf(q)
# date beef veal pork lamb_and_mutton broilers other_chicken turkey
# 0 1967-03-01 00:00:00 1693 65 1136 61 472.0 None 26.5
# 1 1944-12-01 00:00:00 764 146 1013 91 NaN None NaN
# 2 1969-06-01 00:00:00 1666 50 964 42 573.9 None 85.4
# 3 1983-03-01 00:00:00 1892 37 1303 36 1106.2 None 182.7
#################################################
# UNION ALL
#################################################
q = """
SELECT
date
, 'beef' AS meat_type
, beef AS value
FROM meat
UNION ALL
SELECT
date
, 'veal' AS meat_type
, veal AS value
FROM meat
UNION ALL
SELECT
date
, 'pork' AS meat_type
, pork AS value
FROM meat
UNION ALL
SELECT
date
, 'lamb_and_mutton' AS meat_type
, lamb_and_mutton AS value
FROM meat
ORDER BY 1
"""
print pysqldf(q).head(20)
# date meat_type value
# 0 1944-01-01 00:00:00 beef 751
# 1 1944-01-01 00:00:00 veal 85
# 2 1944-01-01 00:00:00 pork 1280
# 3 1944-01-01 00:00:00 lamb_and_mutton 89
#################################################
# subqueries
# fancy!
#################################################
q = """
SELECT
m1.date
, m1.beef
FROM
meat m1
WHERE m1.date IN
(SELECT
date
FROM meat
WHERE
beef >= broilers
ORDER BY date)
"""
more_beef_than_broilers = pysqldf(q)
print more_beef_than_broilers.head(10)
# date beef
# 0 1960-01-01 00:00:00 1196
# 1 1960-02-01 00:00:00 1089
# 2 1960-03-01 00:00:00 1201
# 3 1960-04-01 00:00:00 1066
Final thoughts
pandas is an incredible tool for data analysis in large part, we think, because it is extremely digestible, succinct, and expressive. Ultimately, there are tons of reasons to learn the nuances of merge, join, concatenate, melt and other native pandas features for slicing and dicing data. Check out the docs for some examples.
Our hope is that pandasql will be a helpful learning tool for folks new to Python and pandas. In my own personal experience learning R, sqldf was a familiar interface helping me become highly productive with a new tool as quickly as possible.
# -*- coding:utf-8 -*-
import re
import os
import time
#str.split(string)分割字符串
#’连接符’.join(list) 将列表组成字符串
def change_name(path):
global i
if not os.path.isdir(path) and not os.path.isfile(path):
return False
if os.path.isfile(path):
file_path = os.path.split(path) #分割出目录与文件
lists = file_path[1].split(‘.’) #分割出文件与文件扩展名
file_ext = lists[-1] #取出后缀名(列表切片操作)
img_ext = [‘bmp’,’jpeg’,’gif’,’psd’,’png’,’jpg’]
if file_ext in img_ext:
os.rename(path,file_path[0]+’/’+lists[0]+’_fc.’+file_ext)
i+=1 #注意这里的i是一个陷阱
#或者
#img_ext = ‘bmp|jpeg|gif|psd|png|jpg’
#if file_ext in img_ext:
# print(‘ok—‘+file_ext)
elif os.path.isdir(path):
for x in os.listdir(path):
change_name(os.path.join(path,x)) #os.path.join()在路径处理上很有用
img_dir = ‘D:\\xx\\xx\\images’
img_dir = img_dir.replace(‘\\’,’/’)
start = time.time()
i = 0
change_name(img_dir)
c = time.time() – start
print(‘程序运行耗时:%0.2f’%(c))
print(‘总共处理了 %s 张图片’%(i))
And now, in a break from the previous trend of fluffy posts, we have a tutorial on how to (deep breath): connect PHP to a MSSQL Server 2008 instance over ODBC from Ubuntu Linux using the FreeTDS driver and unixODBC. Theoretically it would also work for SQL Server 2005.
I don’t know whether half of the settings flags are necessary or even correct, but what follows Worked for Me™, YMMV, etc, etc.
In the commands below, I’ll use 192.168.0.1 as the server housing the SQL Server instance, with a SQL Server user name of devuser, password devpass. I’m assuming SQL Server is set up to listen on its default port, 1433. Keep an eye out, because you’ll need to change these things to your own settings.
First, install unixODBC:
sudo apt-get install unixodbc unixodbc-dev
I also installed the following (perhaps necessary) packages: sudo apt-get install tdsodbc php5-odbc
Then download, untar, compile, and install FreeTDS (warning, the URL may change): cd /usr/local
wget http://ibiblio.org/pub/Linux/ALPHA/freetds/stable/freetds-stable.tgz
tar xvfz freetds-stable.tgz
cd freetds-0.82
./configure --enable-msdblib --with-tdsver=8.0 --with-unixodbc=/usr
make
make install
make clean
Attempt a connection over Telnet to your SQL Server instance: telnet 192.168.0.1 1433
Use the tsql tool to test out the connection: tsql -S 192.168.0.1 -U devuser
This should prompt you for the password, after which you can hope against hope to see this beautiful sign: 1>
If that worked, I recommend throwing a (coding) party. Next up is some configging. Open the FreeTDS config file. /usr/local/etc/freetds.conf
Add the following entry to the bottom of the file. We’re setting up a datasource name (DSN) called ‘MSSQL’. [MSSQL]
host = 192.168.0.1
port = 1433
tds version = 8.0
Now open the ODBC configuration file: /usr/local/etc/odbcinst.ini
And add the following MSSQL driver entry (FreeTDS) at the end: [FreeTDS]
Description = FreeTDS driver
Driver = /usr/local/lib/libtdsodbc.so
Setup=/usr/lib/odbc/libtdsS.so
FileUsage = 1
UsageCount = 1
Then, finally, set up the DSN within ODBC in the odbc.ini file here /usr/local/etc/odbc.ini
By adding this bit to the file: [MSSQL]
Description = MS SQL Server
Driver = /usr/local/lib/libtdsodbc.so
Server = 192.168.2.3
UID = devuser
PWD = devpass
ReadOnly = No
Port = 1433
Test out the connection using the isql tool: isql -v MSSQL devuser 'devpass'
If you see “Connected!” you’re golden, congratulations! If not, I’m truly sorry; see below where there are some resources that might help.
Now restart Apache and test it from PHP using ‘MSSQL’ as the DSN. If something doesn’t work, you might try installing any or all of these packages: mdbtools libmdbodbc libmdbtools mdbtools-gmdb
Here are some other resources that were helpful to me through this disastrous journey.
The OS X Terminal opens up a world of powerful UNIX utilities and scripts. If you’re migrating from Linux, you’ll find many familiar commands work the way you expect. But power users often aren’t aware that OS X comes with a number of its own text-based utilities not found on any other operating system. Learning about these Mac-only programs can make you more productive on the command line and help you bridge the gap between UNIX and your Mac.
Update: Thanks to reader feedback, I’ve written about a few more commands in a follow-up post: (And eight hundred more).
1. open
open opens files, directories and applications. Exciting, right? But it really does come in handy as a command-line double-click. For instance, typing:
$ open /Applications/Safari.app/
…will launch Safari as if you had double-clicked its icon in the Finder.
If you point open at a file instead, it will try to load the file with its associated GUI application. open screenshot.png on an image will open that image in Preview. You can set the -a flag to choose the app yourself, or -e to open the file for editing in TextEdit.
Running open on a directory will take you straight to that directory in a Finder window. This is especially useful for bringing up the current directory by typing open .
Remember that the integration between Finder and Terminal goes both ways – if you drag a file from Finder into a Terminal window, its full path gets pasted into the command line.
2. pbcopy and pbpaste
These two commands let you copy and paste text from the command line. Of course, you could also just use your mouse—but the real power of pbcopy and pbpaste comes from the fact that they’re UNIX commands, and that means they benefit from piping, redirection, and the ability to be in scripts in conjunction with other commands. Typing:
$ ls ~ | pbcopy
…will copy a list of files in your home directory to the OS X clipboard. You can easily capture the contents of a file:
$ pbcopy < blogpost.txt
..or do something crazier. This hacked-up script will grab the link of the latest Google doodle and copy it to your clipboard.
Using pbcopy with pipes is a great way to capture the output of a command without having to scroll up and carefully select it. This makes it easy to share diagnostic information. pbcopy and pbpaste can also be used to automate or speed up certain kinds of tasks. For instance, if you want to save email subject lines to a task list, you could copy the subjects from Mail.app and run:
$ pbpaste >> tasklist.txt
3. mdfind
Many a Linux power user has tried to use locate to search for files on a Mac and then quickly discovered that it didn’t work. There’s always the venerable UNIX find command, but OS X comes with its own killer search tool: Spotlight. So why not tap into its power from the command line?
That’s exactly what mdfind does. Anything Spotlight can find, mdfind can find too. That includes the ability to search inside files and metadata.
mdfind comes with a few conveniences that make it stand out from its big blue brother. For instance, the -onlyin flag can restrict the search to a single directory:
$ mdfind -onlyin ~/Documents essay
The mdfind database should stay up to date in the background, but you can also troubleshoot it (as well as Spotlight) using mdutil. If Spotlight isn’t working the way it should, mdutil -E will erase the index and rebuild it from scratch. You can also turn off indexing entirely with mdutil -i off.
4. screencapture
screencapture lets you take many different kinds of screenshots. It’s similar to Grab.app and the keyboard shortcuts cmd + shift + 3 and cmd + shift + 4, except it’s far more flexible. Here are just a few different ways you can use screencapture:
Capture the contents of the screen, including the cursor, and attach the resulting image (named ‘image.png’) to a new Mail message:
$ screencapture -C -M image.png
Select a window using your mouse, then capture its contents without the window’s drop shadow and copy the image to the clipboard:
$ screencapture -c -W
Capture the screen after a delay of 10 seconds and then open the new image in Preview:
$ screencapture -T10 -P image.png
Select a portion of the screen with your mouse, capture its contents, and save the image as a pdf:
$ screencapture -s -t pdf image.pdf
To see more options, type screencapture --help
5. launchctl
launchctl lets you interact with the OS X init script system, launchd. With launch daemons and launch agents, you can control the services that start up when you boot your computer. You can even set up scripts to run periodically or at timed intervals in the background, similar to cron jobs on Linux.
For example, if you’d like to have the Apache web server start automatically when you turn on your Mac, simply type:
Running launchctl list will show you what launch scripts are currently loaded. sudo launchctl unload [path/to/script] will stop and unload running scripts, and adding the -w flag will remove those scripts permanently from your boot sequence. I like to run this one on all the auto-update “helpers” created by Adobe apps and Microsoft Office.
Launchd scripts are stored in the folllowing locations:
To see what goes into a launch agent or daemon, there’s a great blog post by Paul Annesley that walks you through the file format. And if you’d like to learn how to write your own launchd scripts, Apple provides some helpful documentation on their Developer site. There’s also the fantastic Lingon app if you’d prefer to avoid the command line entirely.
6. say
This is a fun one: say converts text to speech, using the same TTS engine OS X uses for VoiceOver. Without any options, say will simply speak whatever text you give it out loud.:
$ say "Never trust a computer you can't lift."
You can also use say to speak the contents of a text file with the -f flag, and you can store the resulting audio clip with the -o flag:
$ say -f mynovel.txt -o myaudiobook.aiff
The say command can be useful in place of console logging or alert sounds in scripts. For instance, you can set up an Automator or Hazel script to do batch file processing and then announce the task’s completion with say.
But the most enjoyable use for say is rather more sinister: if you have ssh access to a friend or coworker’s Mac, you can silently log into their machine and haunt them through the command line. Give ‘em a Siri-ous surprise.
You can set the voice (and language!) used by say by changing the default setting in the Dictation & Speech panel in System Preferences.
7. diskutil
diskutil is a command line interface to the Disk Utility app that comes with OS X. It can do everything its graphical cousin can, but it also has some extra capabilities—such as filling a disk with zeroes or random data. Simply type diskutil list to see the path names of disks and removable media attached to your machine, and then point the command at the volume you want to operate on. Be careful: diskutil can permanently destroy data if it’s used incorrectly.
8. brew
Alright–this isn’t technically a native command. But no OS X power user should be without Homebrew. The website calls it “The missing package manager for OS X,” and that couldn’t be truer. If you’ve ever used apt-get in Linux, you will feel right at home in Homebrew.
brew gives you easy access to thousands of free utilities and libraries from the open source community. For instance, brew install imagemagick will set you up with ImageMagick, a powerful utility that makes it possible to do anything from whipping up animated gifs to converting images between dozens of different types. brew install node will introduce you to NodeJS, the hot new tool for developing and running server-side JavaScript apps.
You can have fun with Homebrew too: brew install archey will get you Archey, a cool little script for displaying your Mac’s specs next to a colourful Apple logo. The selection in Homebrew is huge—and because it’s so easy to create formulas, new packages are being added all the time.
ARCHEY—My command line brings all the boys to the yard.
But the best part about Homebrew? It keeps all its files in a single directory: /usr/local/. That means you can install newer versions of system software, such as python and mysql, without interfering with the built-in equivalents. And if you ever want to get rid of your Homebrew installation, it’s easy to remove.
Edit—May 15: Someone suggested to me that that you should not blindly delete the contents of /usr/local/. That’s a fair precaution. To remove Homebrew safely, use the uninstall script.
By Jonathan Sundqvist, Engineer at Server Density.
Published on the 4th February, 2015.
The industry is hotting up at the moment, and there are more tools than you can shake a stick at. Here lies the most comprehensive list on the Internet (of monitoring tools). Featuring over 80 ways to monitor your machines. Within this article we outline:
Command line tools
Network related
System related monitoring
Log monitoring tools
Infrastructure monitoring tools
It’s hard work monitoring and debugging performance problems, but it’s easier with the right tools at the right time. But how much of your valuable time do you think it would take you to investigate all of these tools and find out which one is best for you?
Why not check out Server Density first, it has a beautiful UI, an api that’s easy to use and alerts that will keep downtime to a minimum.
Top 10 System Monitoring Tools
1. Top
This is a small tool which is pre-installed in many unix systems. When you want an overview of all the processes or threads running in the system: top is a good tool. You can order these processes on different criteria and the default criteria is CPU.
Htop is essentially an enhanced version of top. It’s easier to sort by processes. It’s visually easier to understand and has built in commands for common things you would like to do. Plus it’s fully interactive.
Atop monitors all processes much like top and htop, unlike top and htop however it has daily logging of the processes for long-term analysis. It also shows resource consumption by all processes. It will also highlight resources that have reached a critical load.
Apachetop monitors the overall performance of your apache webserver. It’s largely based on mytop. It displays current number of reads, writes and the overall number of requests processed.
ftptop gives you basic information of all the current ftp connections to your server such as the total amount of sessions, how many are uploading and downloading and who the client is.
mytop is a neat tool for monitoring threads and performance of mysql. It gives you a live look into the database and what queries it’s processing in real time.
powertop helps you diagnose issues that has to do with power consumption and power management. It can also help you experiment with power management settings to achieve the most efficient settings for your server. You switch tabs with the tab key.
iotop checks the I/O usage information and gives you a top-like interface to that. It displays columns on read and write and each row represents a process. It also displays the percentage of time the process spent while swapping in and while waiting on I/O.
ntopng is the next generation of ntop and the tool provides a graphical user interface via the browser for network monitoring. It can do stuff such as: geolocate hosts, get network traffic and show ip traffic distribution and analyze it.
iftop is similar to top, but instead of mainly checking for cpu usage it listens to network traffic on selected network interfaces and displays a table of current usage. It can be handy for answering questions such as “Why on earth is my internet connection so slow?!”.
jnettop visualises network traffic in much the same way as iftop does. It also supports customizable text output and a machine-friendly mode to support further analysis.
BandwidthD tracks usage of TCP/IP network subnets and visualises that in the browser by building a html page with graphs in png. There is a database driven system that supports searching, filtering, multiple sensors and custom reports.
EtherApe displays network traffic graphically, the more talkative the bigger the node. It either captures live traffic or can read it from a tcpdump. The displayed can also be refined using a network filter with pcap syntax.
ethtool is used for displaying and modifying some parameters of the network interface controllers. It can also be used to diagnose Ethernet devices and get more statistics from the devices.
NetHogs breaks down network traffic per protocol or per subnet. It then groups by process. So if there’s a surge in network traffic you can fire up NetHogs and see which process is causing it.
iptraf gathers a variety of metrics such as TCP connection packet and byte count, interface statistics and activity indicators, TCP/UDP traffic breakdowns and station packet and byte counts.
ngrep is grep but for the network layer. It’s pcap aware and will allow to specify extended regular or hexadecimal expressions to match against packets of .
MRTG was orginally developed to monitor router traffic, but now it’s able to monitor other network related things as well. It typically collects every five minutes and then generates a html page. It also has the capability of sending warning emails.
Bmon monitors and helps you debug networks. It captures network related statistics and presents it in human friendly way. You can also interact with bmon through curses or through scripting.
20. traceroute
Traceroute is a built-in tool for displaying the route and measuring the delay of packets across a network.
IPTState allows you to watch where traffic that crosses your iptables is going and then sort that by different criteria as you please. The tool also allows you to delete states from the table.
Darkstat captures network traffic and calculates statistics about usage. The reports are served over a simple HTTP server and gives you a nice graphical user interface of the graphs.
vnStat is a network traffic monitor that uses statistics provided by the kernel which ensures light use of system resources. The gathered statistics persists through system reboots. It has color options for the artistic sysadmins.
24. netstat
Netstat is a built-in tool that displays TCP network connections, routing tables and a number of network interfaces. It’s used to find problems in the network.
25. ss
Instead of using netstat, it’s however preferable to use ss. The ss command is capable of showing more information than netstat and is actually faster. If you want a summary statistics you can use the command ss -s.
Nmap allows you to scan your server for open ports or detect which OS is being used. But you could also use this for SQL injection vulnerabilities, network discovery and other means related to penetration testing.
MTR combines the functionality of traceroute and the ping tool into a single network diagnostic tool. When using the tool it will limit the number hops individual packets has to travel while also listening to their expiry. It then repeats this every second.
Tcpdump will output a description of the contents of the packet it just captured which matches the expression that you provided in the command. You can also save the this data for further analysis.
Justniffer is a tcp packet sniffer. You can choose whether you would like to collect low-level data or high-level data with this sniffer. It also allows you to generate logs in customizable way. You could for instance mimic the access log that apache has.
nmon either outputs the data on screen or saves it in a comma separated file. You can display CPU, memory, network, filesystems, top processes. The data can also be added to a RRD database for further analysis.
Conky monitors a plethora of different OS stats. It has support for IMAP and POP3 and even support for many popular music players! For the handy person you could extend it with your own scripts or programs using Lua.
Glances monitors your system and aims to present a maximum amount of information in a minimum amount of space. It has the capability to function in a client/server mode as well as monitoring remotely. It also has a web interface.
Saidar is a very small tool that gives you basic information about your system resources. It displays a full screen of the standard system resources. The emphasis for saidar is being as simple as possible.
RRDtool is a tool developed to handle round-robin databases or RRD. RRD aims to handle time-series data like CPU load, temperatures etc. This tool provides a way to extract RRD data in a graphical format.
Monit has the capability of sending you alerts as well as restarting services if they run into trouble. It’s possible to perform any type of check you could write a script for with monit and it has a web user interface to ease your eyes.
Linux process explorer is akin to the activity monitor for OSX or the windows equivalent. It aims to be more usable than top or ps. You can view each process and see how much memory usage or CPU it uses.
37. df
df is an abbreviation for disk free and is pre-installed program in all unix systems used to display the amount of available disk space for filesystems which the user have access to.
Dstat aims to be a replacement for vmstat, iostat, netstat and ifstat. It allows you to view all of your system resources in real-time. The data can then be exported into csv. Most importantly dstat allows for plugins and could thus be extended into areas not yet known to mankind.
SNMP is the protocol ‘simple network management protocol’ and the Net-SNMP tool suite helps you collect accurate information about your servers using this protocol.
Incron allows you to monitor a directory tree and then take action on those changes. If you wanted to copy files to directory ‘b’ once new files appeared in directory ‘a’ that’s exactly what incron does.
Monitorix is lightweight system monitoring tool. It helps you monitor a single machine and gives you a wealth of metrics. It also has a built-in HTTP server to view graphs and a reporting mechanism of all metrics.
44. vmstat
vmstat or virtual memory statistics is a small built-in tool that monitors and displays a summary about the memory in the machine.
45. uptime
This small command that quickly gives you information about how long the machine has been running, how many users currently are logged on and the system load average for the past 1, 5 and 15 minutes.
46. mpstat
mpstat is a built-in tool that monitors cpu usage. The most common command is using mpstat -P ALL which gives you the usage of all the cores. You can also get an interval update of the CPU usage.
47. pmap
pmap is a built-in tool that reports the memory map of a process. You can use this command to find out causes of memory bottlenecks.
48. ps
The ps command will give you an overview of all the current processes. You can easily select all processes using the command ps -A
sar is a part of the sysstat package and helps you to collect, report and save different system metrics. With different commands it will give you CPU, memory and I/O usage among other things.
Similar to sar collectl collects performance metrics for your machine. By default it shows cpu, network and disk stats but it collects a lot more. The difference to sar is collectl is able to deal with times below 1 second, it can be fed into a plotting tool directly and collectl monitors processes more extensively.
iostat is also part of the sysstat package. This command is used for monitoring system input/output. The reports themselves can be used to change system configurations to better balance input/output load between hard drives in your machine.
52. free
This is a built-in command that displays the total amount of free and used physical memory on your machine. It also displays the buffers used by the kernel at that given moment.
53. /Proc file system
The proc file system gives you a peek into kernel statistics. From these statistics you can get detailed information about the different hardware devices on your machine. Take a look at the full list of the proc file statistics
GKrellm is a gui application that monitor the status of your hardware such CPU, main memory, hard disks, network interfaces and many other things. It can also monitor and launch a mail reader of your choice.
Gnome system monitor is a basic system monitoring tool that has features looking at process dependencies from a tree view, kill or renice processes and graphs of all server metrics.
GoAccess is a real-time web log analyzer which analyzes the access log from either apache, nginx or amazon cloudfront. It’s also possible to output the data into HTML, JSON or CSV. It will give you general statistics, top visitors, 404s, geolocation and many other things.
Logwatch is a log analysis system. It parses through your system’s logs and creates a report analyzing the areas that you specify. It can give you daily reports with short digests of the activities taking place on your machine.
Much like Logwatch Swatch also monitors your logs, but instead of giving reports it watches for regular expression and notifies you via mail or the console when there is a match. It could be used for intruder detection for example.
MultiTail helps you monitor logfiles in multiple windows. You can merge two or more of these logfiles into one. It will also use colors to display the logfiles for easier reading with the help of regular expressions.
acct or psacct (depending on if you use apt-get or yum) allows you to monitor all the commands a users executes inside the system including CPU and memory time. Once installed you get that summary with the command ‘sa’.
Similar to acct this tool monitors users on your system and allows you to see in real time what commands and processes they are using. It gives you a tree structure of all the processes and so you can see exactly what’s happening.
strace is used to diagnose, debug and monitor interactions between processes. The most common thing to do is making strace print a list of system calls made by the program which is useful if the program does not behave as expected.
DTrace is the big brother of strace. It dynamically patches live running instructions with instrumentation code. This allows you to do in-depth performance analysis and troubleshooting. However, it’s not for the weak of heart as there is a 1200 book written on the topic.
Webmin is a web-based system administration tool. It removes the need to manually edit unix configuration files and lets you manage the system remotely if need be. It has a couple of monitoring modules that you can attach to it.
65. stat
Stat is a built-in tool for displaying status information of files and file systems. It will give you information such as when the file was modified, accessed or changed.
66. ifconfig
ifconfig is a built-in tool used to configure the network interfaces. Behind the scenes network monitor tools use ifconfig to set it into promiscuous mode to capture all packets. You can do it yourself with ifconfig eth0 promiscand return to normal mode with ifconfig eth0 -promisc.
ulimit is a built-in tool that monitors system resources and keeps a limit so any of the monitored resources don’t go overboard. For instance making a fork bomb where a properly configured ulimit is in place would be totally fine.
CPUlimit is a small tool that monitors and then limits the CPU usage of a process. It’s particularly useful to make batch jobs not eat up too many CPU cycles.
69. lshw
lshw is a small built-in tool extract detailed information about the hardware configuration of the machine. It can output everything from CPU version and speed to mainboard configuration.
70. w
W is a built-in command that displays information about the users currently using the machine and their processes.
71. lsof
lsof is a built-in tool that gives you a list of all open files and network connections. From there you can narrow it down to files opened by processes, based on the process name, by a specific user or perhaps kill all processes that belongs to a specific user.
Infrastructure monitoring tools
72. Server Density
Our linux monitoring tool! It has a web interface that allows you to set alerts and view graphs for all system and network metrics. You can also set up monitoring of websites whether they are up or down. Server Density allows you to set permissions for users and you can extend your monitoring with our plugin infrastructure or api. The service already supports Nagios plugins.
OpenNMS has four main functional areas: event management and notifications; discovery and provisioning; service monitoring and data collection. It’s designed to be customizable to work in a variety of network environments.
SysUsage monitors your system continuously via Sar and other system commands. It also allows notifications to alarm you once a threshold is reached. SysUsage itself can be run from a centralized place where all the collected statistics are also being stored. It has a web interface where you can view all the stats.
brainypdm is a data management and monitoring tool that has the capability to gather data from nagios or another generic source to make graphs. It’s cross-platform, has custom graphs and is web based.
PCP has the capability of collating metrics from multiple hosts and does so efficiently. It also has a plugin framework so you can make it collect specific metrics that is important to you. You can access graph data through either a web interface or a GUI. Good for monitoring large systems.
This tool is both a system monitor and task manager. You can view server metrics from several machines through the worksheet and if a process needs to be killed or if you need to start a process it can be done within KDE system guard.
Munin is both a network and a system monitoring tool which offers alerts for when metrics go beyond a given threshold. It uses RRDtool to create the graphs and it has web interface to display these graphs. Its emphasis is on plug and play capabilities with a number of plugins available.
Nagios is system and network monitoring tool that helps you monitor monitor your many servers. It has support for alerting for when things go wrong. It also has many plugins written for the platform.
Zenoss provides a web interface that allows you to monitor all system and network metrics. Moreover it discovers network resources and changes in network configurations. It has alerts for you to take action on and it supports the Nagios plugins.
(And one for luck!) Cacti is network graphing solution that uses the RRDtool data storage. It allows a user to poll services at predetermined intervals and graph the result. Cacti can be extended to monitor a source of your choice through shell scripts.
Zabbix is an open source infrastructure monitoring solution. It can use most databases out there to store the monitoring statistics. The Core is written in C and has a frontend in PHP. If you don’t like installing an agent, Zabbix might be an option for you.
Bonus section:
Thanks for your suggestions. It’s an oversight on our part that we’ll have to go back trough and renumber all the headings. In light of that, here’s a short section at the end for some of the Linux monitoring tools recommended by you:
Collectd is a Unix daemon that collects all your monitoring statistics. It uses a modular design and plugins to fill in any niche monitoring. This way collectd stays as lightweight and customizable as possible.
Observium is an auto-discovering network monitoring platform supporting a wide range of hardware platforms and operating systems. Observium focuses on providing a beautiful and powerful yet simple and intuitive interface to the health and status of your network.
85. Nload
It’s a command line tool that monitors network throughput. It’s neat because it visualizes the in and and outgoing traffic using two graphs and some additional useful data like total amount of transferred data. You can install it with
SmokePing keeps track of the network latencies of your network and it visualises them too. There are a wide range of latency measurement plugins developed for SmokePing. If a GUI is important to you it’s there is an ongoing development to make that happen.
If you’re working in windows environment day in and day out. You may feel limited by the terminal Windows provides. MobaXterm comes to the rescue and allows you to use many of the terminal commands commonly found in Linux. Which will help you tremendously in your monitoring needs!
Shinken is a monitoring framework which is a total rewrite of Nagios in python. It aims to enhance flexibility and managing a large environment. While still keeping all your nagios configuration and plugins.
let words = ["Swift","iOS","cocoa","OSX","tvOS"]
let tweet = "This is an example tweet larking about Swift"
let valid = !words.filter({tweet.containsString($0)}).isEmpty
valid //true
let path = NSBundle.mainBundle().pathForResource("test", ofType: "txt")
let lines = try? String(contentsOfFile: path!).characters.split{$0 == "\n"}.map(String.init)
if let lines=lines {
lines[0] // O! for a Muse of fire, that would ascend
lines[1] // The brightest heaven of invention!
lines[2] // A kingdom for a stage, princes to act
lines[3] // And monarchs to behold the swelling scene.
}
map和字符串构造函数的最后一步把我们的数组字符转换为字符串。
5 祝你生日快乐！
这将显示生日快乐歌到控制台，通过map以及范围和三元运算符的简单使用。
let name = "uraimo"
(1...4).forEach{print("Happy Birthday " + (($0 == 3) ? "dear \(name)":"to You"))}
extension SequenceType{
typealias Element = Self.Generator.Element
func partitionBy(fu: (Element)->Bool)->([Element],[Element]){
var first=[Element]()
var second=[Element]()
for el in self {
if fu(el) {
first.append(el)
}else{
second.append(el)
}
}
return (first,second)
}
}
let part = [82, 58, 76, 49, 88, 90].partitionBy{$0 < 60}
part // ([58, 49], [82, 76, 88, 90])
let xmlDoc = try? AEXMLDocument(xmlData: NSData(contentsOfURL: NSURL(string:"https://www.ibiblio.org/xml/examples/shakespeare/hen_v.xml")!)!)
if let xmlDoc=xmlDoc {
var prologue = xmlDoc.root.children[6]["PROLOGUE"]["SPEECH"]
prologue.children[1].stringValue // Now all the youth of England are on fire,
prologue.children[2].stringValue // And silken dalliance in the wardrobe lies:
prologue.children[3].stringValue // Now thrive the armourers, and honour's thought
prologue.children[4].stringValue // Reigns solely in the breast of every man:
prologue.children[5].stringValue // They sell the pasture now to buy the horse,
}
//Find the minimum of an array of Ints
[10,-22,753,55,137,-1,-279,1034,77].sort().first
[10,-22,753,55,137,-1,-279,1034,77].reduce(Int.max, combine: min)
[10,-22,753,55,137,-1,-279,1034,77].minElement()
//Find the maximum of an array of Ints
[10,-22,753,55,137,-1,-279,1034,77].sort().last
[10,-22,753,55,137,-1,-279,1034,77].reduce(Int.min, combine: max)
[10,-22,753,55,137,-1,-279,1034,77].maxElement()
