You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 17:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Fri Sep 18, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
290092162844,1567sleep029014-21ntp_states07:28:090
20110211907,1166sleep3201-21jbd2/mmcblk2p2-12:07:463
25360210929,1072sleep15718-1kworker/1:1H10:43:131
123742100516,971sleep212434-21/usr/sbin/munin11:43:022
25330249111,462sleep3367-21snmpd10:42:553
1373524679,442sleep030383-1kworker/0:0H08:23:130
1875824298,407sleep318783-21awk08:38:103
2431227721,230sleep12588-21tail09:28:151
27125225123,197sleep327271-21unixbench_singl09:03:183
22747225169,112sleep322769-21awk08:52:443
28471224367,160sleep3289-21dbus-daemon09:08:143
7339222653,27sleep37427-21fschecks_time08:03:053
5784222622,185sleep25875-21fschecks_time07:58:032
5131222414,196sleep05257-21sed11:18:130
21737221220,119sleep321822-21latency_hist10:32:453
18150220420,24sleep30-21swapper/308:37:463
32364220316,164sleep332500-21unixbench_multi07:38:143
16883220122,149sleep116907-21munin-plugin-st11:57:431
8220219210,167sleep38309-21cat11:28:133
31104217817,27sleep30-21swapper/309:18:003
18958217412,142sleep018976-21fschecks_count10:23:010
31862217215,76sleep331978-21apt-config11:02:463
25630217117,139sleep225703-21latency10:43:222
376217054,27sleep30-21swapper/309:23:043
23314216811,142sleep323394-21latency_hist10:37:463
23314216811,142sleep323394-21latency_hist10:37:463
30784216479,29sleep330825-21awk09:17:433
6587216375,24sleep312-21rcu_sched07:58:173
4122216119,16sleep30-21swapper/307:53:003
24054215812,76sleep324139-21mailstats10:38:113
2981215711,133sleep23022-21ntpq07:48:182
9008215515,119sleep30-21swapper/308:08:063
18519215167,61sleep30-21swapper/312:02:463
11224215114,117sleep011299-21mailstats11:38:120
11224215114,117sleep011299-21mailstats11:38:120
3159214943,94sleep33248-21switchtime07:48:233
28332214912,121sleep128334-21sshd12:32:561
25242214711,64sleep30-21swapper/308:58:093
28337214612,31sleep20-21swapper/209:08:092
11998214524,104sleep312022-21sh11:42:433
11998214524,104sleep312022-21sh11:42:433
16962214212,56sleep30-21swapper/311:57:453
26549213649,20sleep30-21swapper/312:27:453
28645213521,90sleep128160-21apt-get10:52:541
30027213417,40sleep330050-21munin-run10:57:453
3302213043,29sleep33333-21grep09:33:023
14221213018,25sleep30-21swapper/308:23:453
17273212913,101sleep117279-21open_files08:33:121
31372212722,91sleep231430-21sed09:18:072
1772521259,26sleep30-21swapper/310:18:083
18576212014,88sleep018639-21/usr/sbin/munin08:38:070
24433211958,18sleep30-21swapper/307:13:143
20792211917,83sleep020856-21mailstats12:08:120
14649211813,88sleep014721-21/usr/sbin/munin11:48:380
26863211717,28sleep226882-21apt-get10:47:552
1153211717,78sleep30-21swapper/311:07:463
31380211642,61sleep30-21swapper/307:37:453
2894021159,90sleep328311-21/usr/sbin/munin12:33:143
16563211543,61sleep316575-21munin-run08:32:453
11925211514,82sleep011928-21apt-get08:17:560
11925211514,82sleep011928-21apt-get08:17:560
7702211414,77sleep07782-21tune2fs11:28:020
22660211420,74sleep022809-21unixbench_singl07:08:150
4864211035,14sleep30-21swapper/309:37:543
4515211033,14sleep30-21swapper/311:17:563
18949211033,14sleep30-21swapper/310:23:003
21058210933,13sleep30-21swapper/312:08:153
2816121089,85sleep31-21systemd10:52:443
1056210831,14sleep30-21swapper/307:43:063
1056210831,14sleep30-21swapper/307:43:063
8198210773,18sleep30-21swapper/309:48:083
8198210773,18sleep30-21swapper/309:48:083
24616210719,51sleep030383-1kworker/0:0H08:57:460
24616210719,51sleep030383-1kworker/0:0H08:57:460
23693210715,73sleep00-21swapper/012:17:570
18584210712,72sleep218597-21modprobe10:22:432
1395821079,83sleep314063-21date10:07:463
30848210615,68sleep130987-21unixbench_singl07:33:151
28630210634,15sleep30-21swapper/307:28:003
13604210614,72sleep013691-21latency_hist11:47:450
12290210632,13sleep30-21swapper/311:42:593
10347210634,12sleep30-21swapper/308:12:583
22486210532,15sleep30-21swapper/312:13:283
9590210433,15sleep30-21swapper/309:53:003
443210414,73sleep0532-21expr11:03:190
15828210420,64sleep215868-21fschecks_count10:13:022
21681210219,65sleep121704-21awk10:32:431
22377210130,12sleep30-21swapper/307:08:103
13707210115,67sleep213205-21apt-get08:22:552
9811210014,72sleep09935-21timerwakeupswit11:33:150
9811210014,72sleep09935-21timerwakeupswit11:33:150
278872996,83sleep327718-21kworker/3:212:30:403
947629611,71sleep19587-21timerwakeupswit08:08:161
2747929614,64sleep30-21swapper/307:23:123
1081429615,65sleep110823-21/usr/sbin/munin11:38:011
1081429615,65sleep110823-21/usr/sbin/munin11:38:011
778529513,66sleep17827-21idleruntime11:28:031
91632949,17sleep30-21swapper/311:32:583
91632949,17sleep30-21swapper/311:32:583
642229312,19sleep30-21swapper/309:42:593
1133029316,15sleep30-21swapper/309:58:053
3056629213,64sleep230575-21df_inode10:58:042
2902729217,55sleep229092-21awk07:28:112
1896429215,57sleep10-21swapper/108:38:161
707129111,65sleep17204-21cat08:02:471
2622629116,58sleep126144-21latency_hist09:02:471
185229113,21sleep00-21swapper/011:08:120
56552897,53sleep10-21swapper/109:38:171
2318528917,54sleep123215-21munin-run07:12:441
177828914,63sleep00-21swapper/007:44:210
177828914,63sleep00-21swapper/007:44:210
613228825,31sleep00-21swapper/009:42:440
3273628714,53sleep2372-21tune2fs09:23:032
217428724,28sleep20-21swapper/207:47:462
217428724,28sleep20-21swapper/207:47:462
2014728726,47sleep30-21swapper/308:43:073
916828612,58sleep29198-21awk08:08:092
148072869,15sleep30-21swapper/310:08:153
143492869,15sleep30-21swapper/311:48:283
325228512,57sleep03372-21timerwakeupswit07:48:240
236912859,14sleep30-21swapper/312:17:563
2065428511,14sleep20-21swapper/210:28:042
1053128516,56sleep110594-21if_eth008:13:051
305828412,59sleep33114-21sed09:32:453
665128316,51sleep30-21swapper/311:23:133
30182837,14sleep30-21swapper/311:13:033
2993828212,57sleep230026-21apt-key10:57:442
2296328212,55sleep223157-21rm08:52:472
2296328212,55sleep223157-21rm08:52:472
97262816,62sleep00-21swapper/008:08:270
311628040,24sleep03258-21latency_hist09:32:460
304572808,53sleep00-21swapper/010:57:500
166528010,53sleep30-21swapper/311:08:073
2969927914,45sleep229489-21/usr/sbin/munin09:13:062
261232799,59sleep326235-21timerwakeupswit07:18:153
2567427913,50sleep125714-21/usr/sbin/munin07:18:051
2034027914,48sleep220407-21/usr/sbin/munin07:03:002
2034027914,48sleep220407-21/usr/sbin/munin07:03:002
1161627925,37sleep011648-21awk08:17:440
48212780,6sleep20-21swapper/207:53:142
1391527816,18sleep10-21swapper/108:23:341
119332788,53sleep30-21swapper/308:17:593
119332788,53sleep30-21swapper/308:17:593
107722786,14sleep20-21swapper/209:57:442
244492776,59sleep3367-21snmpd08:56:393
244492776,59sleep3367-21snmpd08:56:393
2240727721,32sleep20-21swapper/212:13:272
2769327624,29sleep20-21swapper/207:23:152
3011927514,45sleep030244-21sed10:57:460
2649827525,24sleep10-21swapper/112:27:431
1472227519,41sleep114832-21timerandwakeup11:48:391
1241327515,40sleep030791-21kworker/u8:110:02:460
2715727311,47sleep127189-21kernelversion10:48:061
166982737,10sleep30-21swapper/310:13:523
1652627319,33sleep216544-21munin-run08:32:442
2281627219,31sleep122958-21apt-config08:52:461
2054727222,21sleep30-21swapper/307:03:053
2054727222,21sleep30-21swapper/307:03:053
2041627212,17sleep00-21swapper/007:03:020
2041627212,17sleep00-21swapper/007:03:020
74892716,48sleep10-21swapper/108:03:071
357227031,25sleep23661-21/usr/sbin/munin09:33:092
2533727018,29sleep00-21swapper/012:23:020
1682427012,50sleep00-21swapper/011:56:570
159127014,33sleep10-21swapper/111:08:051
1332027013,37sleep213516-21gpgv08:22:472
1332027013,37sleep213516-21gpgv08:22:472
1235627012,40sleep112362-21modprobe10:02:431
499626913,36sleep25060-21/usr/sbin/munin11:18:112
2696326915,38sleep026966-21df_inode10:48:010
2157726916,29sleep221665-21fschecks_time08:48:022
2924126812,41sleep129403-21unixbench_multi07:28:141
2006526813,25sleep20-21swapper/208:43:062
2423226714,32sleep123858-21/usr/sbin/munin07:13:081
2369626718,23sleep00-21swapper/007:12:510
1369426720,22sleep10-21swapper/111:47:461
937126620,26sleep29140-21/usr/sbin/munin11:33:052
937126620,26sleep29140-21/usr/sbin/munin11:33:052
2672626618,32sleep026746-21basename07:22:440
331026516,32sleep03360-21awk11:13:080
3166726536,14sleep00-21swapper/007:37:540
2855326514,27sleep00-21swapper/009:08:150
2234426516,23sleep20-21swapper/210:33:092
2234426516,23sleep20-21swapper/210:33:092
2039626510,15sleep10-21swapper/108:43:131
926126414,29sleep19370-21perl11:33:041
926126414,29sleep19370-21perl11:33:041
856426417,26sleep00-21swapper/008:07:470
2864626418,27sleep128777-21/usr/sbin/munin09:08:161
2647926417,28sleep00-21swapper/009:03:020
2494926420,28sleep225157-21apt-config10:42:462
2085826415,22sleep10-21swapper/112:08:131
1905326413,35sleep219177-21timerwakeupswit08:38:172
1467226420,23sleep10-21swapper/110:08:131
1146126418,28sleep211507-21ntpq09:58:082
68122637,13sleep00-21swapper/009:43:080
385326316,29sleep13914-21/usr/sbin/munin09:33:161
2363526319,22sleep10-21swapper/110:38:001
1769526310,11sleep10-21swapper/111:58:121
1304126315,34sleep113079-21ntpq10:03:081
439126219,19sleep10-21swapper/107:53:061
3136926217,28sleep231378-21modprobe07:37:442
2617026223,23sleep00-21swapper/009:02:460
2210126215,29sleep022095-21apt-get12:12:560
113726222,19sleep20-21swapper/211:07:452
657026112,27sleep20-21swapper/211:23:122
2960026114,24sleep00-21swapper/009:13:040
2535826116,18sleep00-21swapper/010:43:120
252182617,14sleep20-21swapper/208:58:082
2405826121,21sleep10-21swapper/112:18:061
1694726113,32sleep217022-21fw_packets08:33:042
1232826114,33sleep10-21swapper/108:18:091
1232826114,33sleep10-21swapper/108:18:091
927726021,18sleep10-21swapper/109:52:461
927726021,18sleep10-21swapper/109:52:461
737726011,26sleep00-21swapper/011:27:460
178572605,15sleep00-21swapper/008:35:000
618225913,23sleep20-21swapper/209:42:462
49525919,22sleep20-21swapper/207:42:452
3208325914,23sleep10-21swapper/107:38:081
2733325919,19sleep20-21swapper/212:28:132
259025915,18sleep20-21swapper/209:28:162
1898425917,19sleep10-21swapper/110:23:021
183125916,27sleep21838-21ntp_states11:08:102
1814925914,31sleep0231-21systemd-journal11:58:430
961025817,24sleep19719-21perl09:53:021
3240525817,22sleep10-21swapper/109:22:461
32042588,12sleep20-21swapper/211:13:062
2960425817,18sleep10-21swapper/109:13:051
2649425815,27sleep226548-21grep09:03:052
2345325816,24sleep10-21swapper/112:17:461
2268525818,18sleep10-21swapper/110:33:161
2268525818,18sleep10-21swapper/110:33:161
2088125816,21sleep00-21swapper/010:28:100
1548825814,26sleep015545-21irqstats08:28:140
74525711,30sleep032626-21/usr/sbin/munin09:23:130
604125714,26sleep16060-21apt-get11:22:541
3145025713,28sleep031517-21latency_hist07:37:460
2886525718,19sleep20-21swapper/212:33:132
219725711,24sleep00-21swapper/009:28:100
892125614,28sleep28976-21cat11:32:452
83225620,18sleep10-21swapper/107:43:001
83225620,18sleep10-21swapper/107:43:001
388225618,17sleep10-21swapper/107:52:451
3228825613,26sleep132277-21apt-get11:03:001
2020625616,24sleep00-21swapper/008:43:080
2001825619,18sleep00-21swapper/012:07:430
1533625615,26sleep215350-21apt11:52:432
1443225612,28sleep214466-21/usr/sbin/munin10:08:052
669225516,19sleep20-21swapper/209:43:062
65325514,22sleep1659-21ntp_states09:23:101
429825515,18sleep10-21swapper/111:17:461
3046425514,17sleep10-21swapper/110:57:551
2369025515,24sleep223687-21apt-get12:17:552
2031325517,22sleep120318-21sort07:02:551
2031325517,22sleep120318-21sort07:02:551
1935725514,27sleep219432-21mailstats10:23:112
1310625515,27sleep013143-21ps10:03:110
2701325412,25sleep127008-21apt-get07:22:571
2147925411,26sleep021474-21apt-get08:47:560
1720025415,24sleep217196-21apt-get11:57:562
786625317,18sleep20-21swapper/208:03:162
701525311,19sleep00-21swapper/008:02:460
2801725314,22sleep00-21swapper/012:32:220
2522125312,22sleep225230-21munin-run07:17:462
1455825311,26sleep014567-21needreboot10:08:070
1074025312,23sleep210734-21apt-get11:37:582
1074025312,23sleep210734-21apt-get11:37:582
545025212,26sleep00-21swapper/007:57:450
279522525,38sleep00-21swapper/007:23:370
2371525215,22sleep123734-21ntp_states08:53:141
2371525215,22sleep123734-21ntp_states08:53:141
1732625212,24sleep017320-21apt-get10:17:570
1701125212,17sleep00-21swapper/010:17:440
861225117,18sleep00-21swapper/009:48:170
861225117,18sleep00-21swapper/009:48:170
2567625112,21sleep225692-21ntp_states12:23:082
1636725117,18sleep216521-21sed11:53:162
642025013,22sleep16415-21apt-get09:42:571
241432509,24sleep024226-21find10:38:120
1875825011,23sleep118755-21apt-get12:02:561
1169525014,21sleep211771-21sed08:17:462
3110724912,20sleep031176-21df09:18:010
2677724913,16sleep20-21swapper/207:22:462
2199724913,21sleep222004-21df_inode07:08:002
1582024912,21sleep115827-21diskstats10:13:011
996724811,23sleep09973-21ntp_states09:53:090
525824811,14sleep10-21swapper/111:18:141
232202488,29sleep00-21swapper/010:37:350
232202488,29sleep00-21swapper/010:37:350
2076224810,14sleep20-21swapper/212:08:102
2006724814,17sleep20-21swapper/212:07:452
1535824712,18sleep10-21swapper/108:28:111
2766724611,15sleep10-21swapper/109:07:441
127222468,14sleep20-21swapper/210:03:012
1732724514,17sleep10-21swapper/110:17:581
803524410,21sleep28043-21df09:48:032
803524410,21sleep28043-21df09:48:032
2901924412,16sleep00-21swapper/012:33:150
2679324210,13sleep10-21swapper/112:27:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional