You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 18:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed Jun 24, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29172412156,96sleep20-21swapper/212:00:312
2411234211,311sleep22441-21latency_hist08:34:582
2411234211,311sleep22441-21latency_hist08:34:582
991122376,219sleep2364-21snmpd08:56:592
22674223566,20sleep20-21swapper/211:20:202
2746223418,32sleep20-21swapper/208:35:192
2746223418,32sleep20-21swapper/208:35:192
12251223227,192sleep212229-21grep10:49:592
10702223136,179sleep210710-21latency_hist10:44:582
3251222049,156sleep23266-21fschecks_count10:20:152
1872622128,182sleep218843-21gpgv11:09:582
1357721949,169sleep213602-21cpu09:10:122
24188218911,160sleep2225-21systemd-journal09:44:572
18181218511,162sleep218216-21ntpq07:40:272
736621807,19sleep27412-21ntpq12:15:212
31518217815,83sleep20-21swapper/210:05:292
20721217711,146sleep20-21swapper/207:50:002
2928217313,100sleep22973-21latency_hist10:19:582
9898217112,146sleep29903-21cut07:15:312
12964217110,73sleep20-21swapper/207:25:162
27574217010,83sleep20-21swapper/209:54:592
12632216210,56sleep20-21swapper/207:25:002
480021607,137sleep24866-21fschecks_time10:25:152
2155821608,16sleep20-21swapper/207:50:362
15125215931,95sleep10-21swapper/109:15:121
15125215931,95sleep10-21swapper/109:15:121
26466215710,129sleep226588-21timerwakeupswit08:05:302
6052215410,83sleep25756-21/usr/sbin/munin08:45:202
6052215410,83sleep25756-21/usr/sbin/munin08:45:202
752321527,49sleep20-21swapper/207:10:102
9736214811,122sleep0364-21snmpd07:15:130
820214813,120sleep2840-21sh08:29:582
820214813,120sleep2840-21sh08:29:582
5973214618,32sleep36012-21modprobe10:29:573
3217021469,44sleep20-21swapper/211:50:252
6947214512,78sleep27079-21grep10:30:332
1737521429,118sleep217480-21timerwakeupswit09:20:302
1737521429,118sleep217480-21timerwakeupswit09:20:302
1620121426,31sleep20-21swapper/209:17:312
1620121426,31sleep20-21swapper/209:17:312
12047214211,72sleep20-21swapper/212:30:232
1687421405,29sleep20-21swapper/211:01:512
7999213723,39sleep20-21swapper/210:35:172
24686213125,86sleep124209-21apt-get09:45:081
13787213114,35sleep013844-21irqstats09:10:180
433121298,53sleep20-21swapper/212:05:272
21213212821,86sleep321232-21fschecks_count07:50:293
2830921279,88sleep228437-21ls11:39:592
30000212520,81sleep20-21swapper/210:00:282
21134212428,80sleep221283-21latency_hist09:34:592
23918212334,47sleep223979-21grep07:59:572
25014212110,74sleep225031-21uname09:45:422
537921206,36sleep20-21swapper/212:10:042
30653212016,34sleep30-21swapper/310:04:573
25420212021,83sleep20-21swapper/211:30:102
17220212010,97sleep217296-21apt-config07:39:582
127021205,11sleep20-21swapper/210:14:372
127021205,11sleep20-21swapper/210:14:372
12285212013,92sleep312355-21cat09:05:203
12285212013,92sleep312355-21cat09:05:203
10600211916,89sleep310666-21cut12:25:273
2915521189,44sleep20-21swapper/208:15:202
2535621189,93sleep325450-21timerandwakeup09:45:513
873211615,80sleep0942-21latency_hist08:30:000
873211615,80sleep0942-21latency_hist08:30:000
18707211641,36sleep20-21swapper/209:25:222
1199211528,36sleep27331-21cyclictest11:55:262
1199211528,36sleep27331-21cyclictest11:55:262
20385211415,88sleep20-21swapper/211:14:062
15060211322,74sleep10-21swapper/107:30:281
1380121139,28sleep20-21swapper/210:55:002
4316211210,47sleep24346-21fschecks_count08:40:152
4316211210,47sleep24346-21fschecks_count08:40:152
3055221127,52sleep2364-21snmpd11:45:242
18828211212,88sleep118917-21switchtime09:25:251
17497211216,79sleep317610-21users09:20:313
17497211216,79sleep317610-21users09:20:313
11031211235,51sleep311124-21fschecks_time10:45:163
27636211123,29sleep30-21swapper/309:55:003
21603211113,80sleep021626-21/usr/sbin/munin09:35:220
3950211017,80sleep03995-21fschecks_count12:05:170
11403211024,34sleep30-21swapper/307:20:163
1343821089,15sleep00-21swapper/007:25:260
8553210614,75sleep18620-21df12:20:121
2828221065,50sleep20-21swapper/208:10:562
2394210614,74sleep02398-21/usr/sbin/munin12:00:160
2886621058,87sleep2364-21snmpd09:57:212
26536210314,71sleep026630-21memory09:50:200
1009721037,84sleep210093-21apt-get12:25:102
3168210220,68sleep33233-21ls10:20:123
12419210227,63sleep212465-21ntpq09:05:232
12419210227,63sleep212465-21ntpq09:05:232
32380210118,67sleep332388-21needreboot08:25:223
32380210118,67sleep332388-21needreboot08:25:223
3150421015,85sleep20-21swapper/208:22:062
29286210128,48sleep029421-21unixbench_singl11:40:290
29286210128,48sleep029421-21unixbench_singl11:40:290
17748210116,69sleep117238-21apt-get07:40:091
423129915,67sleep10-21swapper/108:40:101
423129915,67sleep10-21swapper/108:40:101
255032997,38sleep225547-21systemd-run08:04:582
849829729,46sleep18655-21unixbench_multi10:35:281
83132977,12sleep00-21swapper/008:51:200
704129718,62sleep07123-21fschecks_time12:15:150
662729738,36sleep06796-21grep07:05:280
662729738,36sleep06796-21grep07:05:280
2619129718,58sleep126319-21/usr/sbin/munin11:30:291
897029612,69sleep09044-21fschecks_time08:55:160
2955129417,62sleep329668-21sort08:15:293
1825229423,39sleep00-21swapper/009:25:080
2705129310,53sleep30-21swapper/311:35:153
1469729310,42sleep20-21swapper/212:40:002
1469729310,42sleep20-21swapper/212:40:002
902729216,53sleep09248-21sed07:15:000
2396029116,54sleep024015-21fschecks_count11:25:150
1998629115,59sleep10-21swapper/109:30:181
1339829118,53sleep313167-21/usr/sbin/munin12:35:173
1339829118,53sleep313167-21/usr/sbin/munin12:35:173
2207729012,38sleep20-21swapper/209:35:322
781528913,58sleep30-21swapper/310:35:103
1488728817,58sleep114966-21cat09:14:591
1425128820,15sleep20-21swapper/210:55:162
631228713,59sleep26361-21ntp_offset07:05:222
631228713,59sleep26361-21ntp_offset07:05:222
316528710,58sleep00-21swapper/010:20:090
312862870,6sleep30-21swapper/308:20:293
2162228717,26sleep10-21swapper/107:50:391
400828618,52sleep34150-21latency_hist08:40:003
400828618,52sleep34150-21latency_hist08:40:003
2261028524,42sleep322673-21lockfile-remove09:39:573
104582828,36sleep20-21swapper/209:00:102
251352817,41sleep225164-21munin-run11:29:572
986128019,28sleep00-21swapper/012:25:000
219202805,62sleep20-21swapper/211:18:512
1441128018,47sleep20-21swapper/207:30:092
1249028019,46sleep012555-21cpuspeed10:50:130
101142808,16sleep10-21swapper/110:40:291
652027920,25sleep30-21swapper/307:05:273
652027920,25sleep30-21swapper/307:05:273
3226827925,26sleep10-21swapper/110:10:001
2363627927,26sleep00-21swapper/011:24:590
1411927932,25sleep00-21swapper/010:55:140
517027828,27sleep10-21swapper/112:09:591
1612827822,24sleep00-21swapper/011:00:270
1533927813,50sleep315391-21munin-run10:59:583
3270927712,51sleep132710-21sshd11:52:341
2265427716,28sleep30-21swapper/307:55:103
902927617,45sleep20-21swapper/212:20:262
758127621,25sleep00-21swapper/010:34:590
2468527513,46sleep024209-21apt-get09:45:070
436927412,48sleep34445-21systemctl12:05:283
2822927420,29sleep00-21swapper/011:39:570
964727320,29sleep18820-21apt-get07:15:091
842127317,20sleep00-21swapper/010:35:270
36142738,47sleep10-21swapper/112:04:581
1134127316,42sleep111368-21/usr/sbin/munin10:45:221
858127218,43sleep20-21swapper/208:54:452
3140327220,23sleep00-21swapper/011:49:580
1737927215,32sleep017546-21sed07:40:000
559127115,32sleep30-21swapper/312:10:173
2859727125,28sleep0296-21rs:main0
2638427119,25sleep10-21swapper/109:50:171
1649127122,22sleep00-21swapper/009:19:590
1649127122,22sleep00-21swapper/009:19:590
2971827018,43sleep20-21swapper/211:43:512
2971827018,43sleep20-21swapper/211:43:512
2437927020,29sleep10-21swapper/109:45:001
2101027012,43sleep321103-21cut11:15:213
203242709,51sleep00-21swapper/007:46:380
168582708,12sleep30-21swapper/307:35:313
965326911,43sleep39665-21/usr/sbin/munin10:40:193
2330226921,27sleep30-21swapper/309:40:263
206826927,26sleep30-21swapper/311:59:593
206826927,26sleep30-21swapper/311:59:593
1932326915,41sleep319354-21cpu07:45:123
134526920,23sleep30-21swapper/310:14:593
134526920,23sleep30-21swapper/310:14:593
915226828,21sleep30-21swapper/308:55:193
583226816,35sleep05861-21ntp_states12:10:220
2430726816,37sleep324316-21/usr/sbin/munin08:00:143
1299426816,22sleep30-21swapper/312:35:003
2123726722,28sleep021327-21fschecks_time07:50:300
2054426714,34sleep020653-21/usr/sbin/munin09:30:290
1798126721,26sleep00-21swapper/011:05:280
3013926612,40sleep330166-21df11:45:133
245626614,35sleep32504-21fschecks_time12:00:193
671926515,34sleep16853-21sed12:14:591
205826515,27sleep10-21swapper/110:15:251
1441626518,28sleep014410-21apt-get07:30:110
1014226514,35sleep110189-21modprobe08:59:571
831726419,22sleep10-21swapper/112:20:001
591526428,17sleep10-21swapper/107:05:121
591526428,17sleep10-21swapper/107:05:121
314826415,30sleep00-21swapper/008:35:290
314826415,30sleep00-21swapper/008:35:290
3107426414,34sleep131148-21find08:20:281
2316426424,21sleep30-21swapper/311:20:293
2049226412,36sleep120501-21munin-run11:14:581
1916126417,29sleep318679-21apt-get11:10:073
1600326424,22sleep10-21swapper/111:00:241
1248626419,29sleep112483-21apt-get10:50:101
840526320,21sleep30-21swapper/308:52:323
803326318,27sleep38095-21mailstats07:10:273
708226318,26sleep37168-21users10:30:343
471826313,37sleep04781-21sendmail_mailst08:40:260
471826313,37sleep04781-21sendmail_mailst08:40:260
2817926319,29sleep028190-21/usr/sbin/munin09:55:210
2334026314,21sleep00-21swapper/009:40:270
1380326324,23sleep10-21swapper/112:35:271
1380326324,23sleep10-21swapper/112:35:271
1201826311,27sleep00-21swapper/009:05:130
1201826311,27sleep00-21swapper/009:05:130
2007426221,23sleep30-21swapper/309:30:203
1483626218,25sleep314882-21systemctl09:14:573
1135526214,29sleep00-21swapper/012:29:570
1000526217,22sleep00-21swapper/010:40:280
97982617,15sleep30-21swapper/307:15:163
516126116,25sleep10-21swapper/110:25:231
830226010,14sleep30-21swapper/312:19:583
3243926019,24sleep132472-21sshd08:25:241
3243926019,24sleep132472-21sshd08:25:241
185226013,28sleep31946-21/usr/sbin/munin10:15:193
1787726011,10sleep00-21swapper/009:23:470
1787726011,10sleep00-21swapper/009:23:470
2980325915,26sleep30-21swapper/311:44:573
2980325915,26sleep30-21swapper/311:44:573
269742599,10sleep00-21swapper/008:09:000
2401825913,32sleep124083-21fw_packets11:25:161
1921525919,24sleep019182-21apt-get11:10:290
1553625916,25sleep115731-21latency_hist11:00:001
134725918,18sleep30-21swapper/308:30:193
134725918,18sleep30-21swapper/308:30:193
1258025916,29sleep012613-21munin-run07:24:580
1185325916,20sleep10-21swapper/107:20:271
720825821,23sleep10-21swapper/108:50:041
720825821,23sleep10-21swapper/108:50:041
48625815,19sleep00-21swapper/010:10:230
48625815,19sleep00-21swapper/010:10:230
3072125817,24sleep130838-21grep11:45:271
2697325822,16sleep10-21swapper/111:35:131
263325816,24sleep10-21swapper/108:35:091
263325816,24sleep10-21swapper/108:35:091
176962586,12sleep10-21swapper/109:21:301
176962586,12sleep10-21swapper/109:21:301
2876425714,25sleep128813-21irqstats11:40:181
2876425714,25sleep128813-21irqstats11:40:181
2733625716,25sleep127333-21apt-get08:10:101
2399825718,20sleep00-21swapper/007:59:590
1374625720,23sleep313776-21munin-run10:54:573
3113225613,18sleep00-21swapper/010:05:180
3091825613,26sleep330937-21apt-get10:05:093
2928525616,20sleep10-21swapper/108:15:231
2687925617,18sleep10-21swapper/108:07:481
2432325617,21sleep10-21swapper/108:00:161
180325613,26sleep11804-21sshd11:57:351
180325613,26sleep11804-21sshd11:57:351
119825616,16sleep10-21swapper/108:30:171
119825616,16sleep10-21swapper/108:30:171
763125513,29sleep17661-21grep07:10:171
584625515,25sleep05857-21df_inode08:45:140
584625515,25sleep05857-21df_inode08:45:140
3172125515,17sleep30-21swapper/311:50:143
2676625518,22sleep326912-21sed11:34:593
2196725519,16sleep10-21swapper/109:35:311
11392556,10sleep10-21swapper/110:12:591
11392556,10sleep10-21swapper/110:12:591
1091925518,19sleep30-21swapper/309:00:243
73125419,17sleep00-21swapper/011:55:120
73125419,17sleep00-21swapper/011:55:120
671825415,23sleep16720-21ntp_states10:30:281
647625418,18sleep30-21swapper/308:45:293
647625418,18sleep30-21swapper/308:45:293
305012547,11sleep10-21swapper/110:03:011
2710025416,23sleep327223-21wc08:10:003
2517625411,26sleep325185-21munin-run11:29:583
1844725419,16sleep00-21swapper/007:40:330
1835925412,27sleep318426-21switchtime07:40:323
1738125414,24sleep117400-21apt-get11:05:091
168532545,36sleep30-21swapper/311:01:353
1646325418,20sleep30-21swapper/309:19:583
1646325418,20sleep30-21swapper/309:19:583
1335325418,15sleep30-21swapper/307:25:253
1248425419,19sleep10-21swapper/109:05:261
1248425419,19sleep10-21swapper/109:05:261
1093725416,22sleep010933-21apt-get10:45:090
459425316,17sleep30-21swapper/308:40:203
459425316,17sleep30-21swapper/308:40:203
324425313,16sleep10-21swapper/110:20:141
2888225314,21sleep028879-21apt-get08:15:100
2423625313,23sleep024231-21apt-get08:00:110
2292925318,19sleep10-21swapper/111:20:271
1173225319,20sleep00-21swapper/012:30:170
2945725215,24sleep029465-21df10:00:140
2846825210,32sleep30-21swapper/308:13:193
2634125210,24sleep30-21swapper/309:50:153
2073725213,23sleep120734-21apt-get11:15:101
182292529,17sleep30-21swapper/309:25:073
1332125214,22sleep113327-21ntp_states07:25:231
105282529,15sleep00-21swapper/009:00:120
471325112,22sleep04708-21apt-get10:25:090
2073125110,17sleep00-21swapper/011:15:090
204225114,23sleep02057-21open_inodes10:15:240
1481225115,22sleep314850-21ntpq07:30:223
1164825116,16sleep10-21swapper/112:30:141
1089125113,23sleep110910-21ntp_states09:00:221
3175325012,22sleep00-21swapper/011:50:150
2781125013,23sleep127808-21apt-get09:55:101
1973725010,22sleep119774-21ntp_states07:45:221
3068124910,15sleep00-21swapper/008:20:180
257142497,16sleep00-21swapper/011:30:200
1360924910,19sleep00-21swapper/012:35:220
1360924910,19sleep00-21swapper/012:35:220
1659424714,16sleep10-21swapper/107:35:291
138224713,14sleep00-21swapper/008:30:200
138224713,14sleep00-21swapper/008:30:200
794424615,18sleep10-21swapper/108:50:281
2315924619,15sleep10-21swapper/107:55:241
1073024514,14sleep10-21swapper/112:25:291
1780524412,15sleep30-21swapper/311:05:233
977324110,13sleep30-21swapper/312:24:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional