You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-09 - 09:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 09, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
313282616010,6137sleep025263-21kworker/0:122:20:190
241202118012,1151sleep316613-1kworker/3:0H21:59:183
142052106625,1027sleep214211-21df_inode19:45:182
142052106625,1027sleep214211-21df_inode19:45:182
1936252530,478sleep11937-21ntp_states00:14:291
3184523576,319sleep2364-21snmpd00:05:542
27504235615,324sleep227540-21latency22:09:272
89992299131,150sleep29129-21grep21:09:362
80812295125,48sleep24813-21kworker/2:221:09:072
174652265117,36sleep217516-21cpuspeed19:55:112
13526225854,184sleep213543-21awk19:44:042
20288223830,192sleep220404-21latency_hist20:04:072
8905223723,18sleep20-21swapper/219:29:072
8905223723,18sleep20-21swapper/219:29:072
28062223164,22sleep20-21swapper/220:29:062
19832221218,176sleep219939-21sort21:44:362
14771219816,159sleep214858-21latency_hist23:14:062
4675218538,80sleep20-21swapper/219:14:252
30309218219,141sleep20-21swapper/220:34:312
47172181145,16sleep20-21swapper/222:39:342
47172181145,16sleep20-21swapper/222:39:342
32120218116,140sleep20-21swapper/222:24:242
16177218116,151sleep216256-21switchtime19:49:342
11449217813,153sleep111456-21grep21:19:191
11449217813,153sleep111456-21grep21:19:191
10445217612,149sleep010424-21latency_hist19:34:050
10445217612,149sleep010424-21latency_hist19:34:050
18710217421,126sleep20-21swapper/223:24:322
126521747,15sleep00-21swapper/019:04:270
2368021738,147sleep20-21swapper/223:39:382
470721729,105sleep20-21swapper/200:24:232
42012169120,25sleep20-21swapper/220:54:342
21183216715,19sleep20-21swapper/220:04:362
1527021679,124sleep215284-21grep23:14:262
78442152110,19sleep20-21swapper/222:49:352
21226215133,102sleep221260-21ps21:49:322
15972215113,104sleep00-21swapper/019:49:270
14616215111,18sleep20-21swapper/221:29:222
121621508,131sleep2983-21/usr/sbin/munin22:29:252
29499214311,119sleep229507-21awk23:59:322
2332021427,13sleep20-21swapper/221:55:502
1145721429,30sleep20-21swapper/221:19:202
1145721429,30sleep20-21swapper/221:19:202
2007821419,95sleep220105-21ntpq23:29:282
16178214016,19sleep20-21swapper/221:34:222
2473213618,34sleep22275-21cyclictest20:49:292
2235213655,39sleep30-21swapper/319:09:063
22195213614,80sleep20-21swapper/220:09:242
17484213213,16sleep217631-21modprobe21:39:072
10384213214,82sleep20-21swapper/219:33:512
10384213214,82sleep20-21swapper/219:33:512
25446212914,15sleep20-21swapper/222:04:062
25948212717,82sleep026098-21unixbench_singl20:19:350
24369212615,87sleep00-21swapper/022:00:040
2165421217,72sleep20-21swapper/223:34:292
7428212085,21sleep27466-21modprobe00:34:042
3155321206,38sleep20-21swapper/222:23:112
16689212075,26sleep216796-21grep23:19:232
14045211818,82sleep113541-21apt-get19:44:161
14045211818,82sleep113541-21apt-get19:44:161
1166721188,11sleep20-21swapper/219:35:252
2468621178,11sleep20-21swapper/220:16:262
604921167,93sleep20-21swapper/219:19:182
151821157,33sleep20-21swapper/220:45:412
1103321157,35sleep20-21swapper/221:17:472
1103321157,35sleep20-21swapper/221:17:472
7019211413,85sleep07034-21kernelversion21:04:250
30861211421,71sleep00-21swapper/000:04:380
13626211311,17sleep20-21swapper/221:24:332
13626211311,17sleep20-21swapper/221:24:332
792421126,29sleep20-21swapper/200:34:242
2683821126,32sleep20-21swapper/222:07:412
2562321129,64sleep20-21swapper/220:19:292
18327211111,84sleep10-21swapper/123:24:221
1056721117,62sleep20-21swapper/222:59:252
1301221106,29sleep20-21swapper/223:06:592
642621096,28sleep20-21swapper/221:03:302
407421085,28sleep20-21swapper/200:20:472
301921076,60sleep20-21swapper/222:34:302
301921076,60sleep20-21swapper/222:34:302
28969210610,15sleep20-21swapper/222:14:232
11156210624,66sleep011220-21processes19:34:300
2653210468,22sleep22722-21basename19:09:252
13729210015,64sleep113793-21cat23:09:251
2870629915,65sleep328745-21/usr/sbin/munin20:29:323
274472997,13sleep20-21swapper/223:54:152
1819529916,62sleep318339-21unixbench_singl19:55:283
1289429915,27sleep10-21swapper/119:39:351
3219929713,65sleep032277-21/usr/sbin/munin22:24:260
624929613,66sleep06256-21diskstats00:29:210
92452957,11sleep20-21swapper/222:54:322
583829411,70sleep05908-21idleruntime22:44:250
583829411,70sleep05908-21idleruntime22:44:250
269482945,12sleep10-21swapper/123:51:191
1824129423,46sleep018325-21fschecks_time23:24:210
269342937,11sleep20-21swapper/223:51:082
2444229315,64sleep324465-21fschecks_count23:44:233
1662629325,43sleep016682-21mailstats21:34:320
338529218,59sleep33353-21latency_hist20:54:063
326622929,10sleep20-21swapper/220:43:482
323802928,16sleep20-21swapper/200:09:172
282682926,13sleep20-21swapper/220:29:102
243282926,11sleep20-21swapper/223:44:142
1704629215,60sleep117052-21ntp_states23:19:301
24512916,12sleep20-21swapper/200:15:182
2877929013,62sleep128824-21grep22:14:181
210202909,14sleep00-21swapper/020:04:340
177882900,11sleep30-21swapper/321:39:233
886928919,53sleep18886-21fschecks_count22:54:231
614328912,47sleep20-21swapper/222:44:312
614328912,47sleep20-21swapper/222:44:312
922828824,30sleep30-21swapper/319:29:243
922828824,30sleep30-21swapper/319:29:243
413928811,34sleep30-21swapper/322:39:193
413928811,34sleep30-21swapper/322:39:193
724028718,52sleep37303-21/usr/sbin/munin22:49:203
1317128723,44sleep013187-21cron23:09:030
3062828618,46sleep330337-21/usr/sbin/munin22:19:253
274028615,54sleep02860-21timerwakeupswit20:49:350
730928514,50sleep07311-21cron19:24:040
2406928516,55sleep124117-21unin-run23:44:051
3114328420,41sleep130337-21/usr/sbin/munin22:19:351
1743028422,36sleep10-21swapper/121:39:061
897428326,39sleep19006-21sendmail00:39:031
3217628313,50sleep332196-21kernelversion22:24:253
2023228126,38sleep319661-21/usr/sbin/munin23:29:333
1822228117,41sleep318235-21fschecks_count23:24:203
2257028023,27sleep30-21swapper/323:39:063
2210028014,33sleep00-21swapper/021:54:060
433427920,40sleep34361-21latency_hist00:24:033
3118027924,41sleep031218-21latency_hist20:39:050
2447027916,48sleep024559-21grep23:44:240
2130027917,41sleep320500-21/usr/sbin/munin20:04:373
1317927918,32sleep00-21swapper/021:24:230
1317927918,32sleep00-21swapper/021:24:230
75932789,49sleep00-21swapper/019:24:160
75932789,49sleep00-21swapper/019:24:160
1537727819,25sleep00-21swapper/023:14:280
1275827824,29sleep30-21swapper/321:24:063
1275827824,29sleep30-21swapper/321:24:063
905427716,26sleep30-21swapper/300:39:053
3142427719,46sleep031429-21cpu20:39:180
237452765,13sleep30-21swapper/320:14:263
220692765,20sleep30-21swapper/320:09:173
1122227618,39sleep111291-21sensors_temp19:34:311
71427517,36sleep1729-21idleruntime-cro22:29:041
2753827520,27sleep10-21swapper/123:54:221
1634727521,24sleep10-21swapper/123:19:061
872627417,32sleep00-21swapper/021:09:290
376327418,34sleep13787-21awk19:14:041
3169327428,25sleep331752-21latency20:39:253
3032527416,36sleep330389-21grep20:34:323
2716327423,31sleep327177-21apt23:54:043
2264827414,19sleep00-21swapper/020:09:340
926727322,26sleep30-21swapper/322:54:333
2911127317,40sleep329117-21df_inode23:59:213
953927220,26sleep10-21swapper/119:29:301
953927220,26sleep10-21swapper/119:29:301
417227221,26sleep30-21swapper/319:14:103
3275227212,47sleep0323-21ntpq00:09:270
2342427213,36sleep00-21swapper/023:39:360
206827211,17sleep10-21swapper/120:49:201
110627220,21sleep10-21swapper/122:29:241
804927117,37sleep18056-21/usr/sbin/munin19:24:301
804927117,37sleep18056-21/usr/sbin/munin19:24:301
642527124,32sleep16025-21/usr/sbin/munin19:19:291
25732718,13sleep00-21swapper/022:34:190
25732718,13sleep00-21swapper/022:34:190
1167727112,44sleep311808-21latency_hist23:04:063
113627118,35sleep21189-21irqstats19:04:252
109127121,32sleep1896-21/usr/sbin/munin19:04:241
2770327015,38sleep027766-21ps22:09:330
2694327021,22sleep00-21swapper/020:24:240
1025727016,38sleep310293-21ntpq21:14:293
1025727016,38sleep310293-21ntpq21:14:293
933626916,21sleep00-21swapper/022:54:340
434826913,27sleep10-21swapper/122:39:251
434826913,27sleep10-21swapper/122:39:251
2723526919,25sleep30-21swapper/322:09:193
2204626913,39sleep122054-21/usr/sbin/munin20:09:151
1735026921,31sleep317351-21cron21:39:033
1282126918,28sleep00-21swapper/019:39:340
389326817,25sleep10-21swapper/122:39:051
389326817,25sleep10-21swapper/122:39:051
2488726818,33sleep124912-21munin-run20:19:041
2076326810,34sleep10-21swapper/121:49:191
2052726819,25sleep10-21swapper/121:49:071
91826710,12sleep30-21swapper/320:44:293
68942679,45sleep3364-21snmpd22:48:233
68942679,45sleep3364-21snmpd22:48:233
62826716,21sleep30-21swapper/300:09:343
2805626715,34sleep028142-21ls23:54:330
2485626727,21sleep10-21swapper/123:44:331
2368426720,31sleep323827-21cmp21:59:063
2356926720,28sleep123599-21apt-get21:59:041
1994026715,35sleep320074-21unixbench_singl21:44:373
1077526717,32sleep310811-21/usr/sbin/munin19:34:213
2206526622,23sleep30-21swapper/321:54:053
1707226615,20sleep30-21swapper/319:54:073
483526519,21sleep10-21swapper/119:14:281
462526516,26sleep30-21swapper/300:24:203
2854326520,22sleep10-21swapper/122:14:061
1440226522,23sleep30-21swapper/319:45:233
1440226522,23sleep30-21swapper/319:45:233
1421826525,20sleep00-21swapper/019:45:200
1421826525,20sleep00-21swapper/019:45:200
1318826514,36sleep113222-21munin-run23:09:041
613626414,21sleep30-21swapper/319:19:233
45726419,26sleep10-21swapper/100:09:321
362626425,23sleep00-21swapper/020:54:190
1774426414,30sleep017750-21/usr/sbin/munin21:39:210
1491426412,37sleep114946-21awk21:29:291
962726312,24sleep10-21swapper/121:14:071
749226316,20sleep10-21swapper/100:34:071
327426317,32sleep13329-21if_eth000:19:251
3214526320,20sleep30-21swapper/300:09:063
2728726313,23sleep10-21swapper/120:24:331
1973126313,21sleep10-21swapper/119:59:381
158426320,21sleep30-21swapper/322:29:343
575226223,21sleep10-21swapper/100:27:241
547226217,24sleep30-21swapper/320:59:353
495226211,37sleep14961-21systemd-run20:59:051
2171726211,24sleep10-21swapper/123:34:321
2079526223,22sleep10-21swapper/120:04:261
2030226220,25sleep020431-21timerwakeupswit23:29:340
1969626225,16sleep00-21swapper/021:44:340
381326114,30sleep33909-21latency_hist19:14:063
2023726126,21sleep00-21swapper/020:04:050
1898026117,27sleep119066-21grep21:44:061
59926018,22sleep10-21swapper/120:44:231
455326014,20sleep00-21swapper/019:14:230
2871026020,22sleep328743-21munin-run23:59:043
2120826015,27sleep021227-21apt-get23:34:140
1354926013,26sleep30-21swapper/319:44:053
1036026018,25sleep10-21swapper/122:59:191
533525914,20sleep10-21swapper/120:59:331
317025921,23sleep03273-21cut00:19:240
308622598,22sleep10-21swapper/100:04:401
2456625923,17sleep10-21swapper/122:00:091
1785125914,28sleep317876-21munin-run23:24:043
1399325917,27sleep014045-21tr23:09:320
34912585,11sleep00-21swapper/019:10:320
2584825813,25sleep025906-21/usr/sbin/munin22:04:240
1754825815,23sleep10-21swapper/119:55:141
76722575,14sleep10-21swapper/121:04:381
705425716,23sleep17197-21modprobe22:49:061
3011825716,28sleep130192-21cat20:34:261
2718425710,31sleep027214-21munin-run23:54:050
146322578,30sleep00-21swapper/021:29:240
796525617,22sleep30-21swapper/319:24:273
796525617,22sleep30-21swapper/319:24:273
2614225613,27sleep326148-21ntp_states22:04:313
2568925614,27sleep125750-21sendmail_mailst20:19:321
675425514,25sleep36751-21apt-get21:04:163
566925510,28sleep05675-21ntp_states20:59:400
2934425511,31sleep129345-21sshd20:31:191
2912225512,27sleep129124-21diskstats23:59:231
128225510,18sleep00-21swapper/022:29:270
114222556,12sleep00-21swapper/023:01:240
1124625512,19sleep00-21swapper/021:19:060
986425416,23sleep09858-21apt-get21:14:180
72642545,40sleep30-21swapper/300:31:593
3043425419,18sleep00-21swapper/000:04:060
2684525418,22sleep326936-21fschecks_time20:24:233
2005425413,26sleep120077-21/usr/sbin/munin23:29:271
1746325412,23sleep017382-21apt-get19:55:090
592125311,23sleep00-21swapper/000:29:050
3179325318,20sleep10-21swapper/120:39:271
1562425318,18sleep30-21swapper/323:14:353
1044125314,21sleep10-21swapper/121:14:351
1044125314,21sleep10-21swapper/121:14:351
834425214,22sleep38391-21df21:09:203
768925212,22sleep07708-21apt-get00:34:140
261192525,11sleep30-21swapper/320:19:393
1207725216,17sleep00-21swapper/021:19:350
1207725216,17sleep00-21swapper/021:19:350
566125117,18sleep10-21swapper/122:44:131
566125117,18sleep10-21swapper/122:44:131
371025116,17sleep30-21swapper/320:54:223
253825113,23sleep32557-21apt-get22:34:143
253825113,23sleep32557-21apt-get22:34:143
2303125111,16sleep30-21swapper/323:39:263
143425111,18sleep00-21swapper/020:44:390
1354925112,19sleep10-21swapper/121:24:321
1354925112,19sleep10-21swapper/121:24:321
1345425115,21sleep313452-21apt-get23:09:153
2957325012,22sleep029608-21munin-run20:34:050
2698825013,14sleep10-21swapper/122:09:051
1560125013,21sleep115598-21apt-get19:49:161
292092497,16sleep30-21swapper/322:14:283
2359324912,20sleep00-21swapper/020:14:140
147424912,20sleep00-21swapper/000:14:150
147224912,21sleep31469-21apt-get00:14:143
413124812,21sleep04134-21apt-get22:39:160
413124812,21sleep04134-21apt-get22:39:160
1702124812,21sleep017045-21ntp_states23:19:290
155292484,32sleep30-21swapper/321:30:323
201202477,31sleep30-21swapper/320:02:353
1296824712,20sleep30-21swapper/321:24:143
1296824712,20sleep30-21swapper/321:24:143
2877824611,20sleep028772-21apt-get22:14:170
854624511,16sleep08590-21wc22:54:050
1191324510,18sleep30-21swapper/323:04:183
2231024411,16sleep00-21swapper/021:54:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional