You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-05 - 05:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Jun 05, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
223942752716,7496sleep210075-21kworker/2:219:43:372
260752416511,4142sleep224389-21kworker/2:121:36:372
20299211526,1134sleep118718-21kworker/1:023:00:571
16926281111,789sleep324447-1kworker/3:0H21:08:313
23415275117,702sleep131773-1kworker/1:0H21:28:441
23415275117,702sleep131773-1kworker/1:0H21:28:441
5352244828,395sleep15415-40mandb23:58:091
5352244828,395sleep15415-40mandb23:58:091
620323776,361sleep1363-21snmpd22:15:111
21494231448,244sleep121630-21unixbench_singl23:03:401
38822698,246sleep1334-21latency_hist20:18:111
29608226584,132sleep129676-21latency_hist21:48:101
31660225810,98sleep10-21swapper/123:38:231
13322225437,136sleep113469-21unixbench_singl19:13:401
23941223613,208sleep023953-21fschecks_count23:13:280
12084222262,56sleep10-21swapper/122:33:401
12084222262,56sleep10-21swapper/122:33:401
10411221068,118sleep10-21swapper/120:48:191
2220321779,154sleep222278-21find21:23:412
2220321779,154sleep222278-21find21:23:412
2356921767,133sleep123647-21fschecks_time19:48:281
4880217338,123sleep11-21systemd22:13:091
31817216823,48sleep10-21swapper/121:53:351
20977216711,15sleep30-21swapper/323:03:303
31008216412,135sleep131087-21cmp20:13:111
31008216412,135sleep131087-21cmp20:13:111
2155216316,125sleep12247-21fschecks_time22:03:291
20541215213,15sleep10-21swapper/121:18:391
2548215016,108sleep22531-21apt-get23:48:302
2976321477,128sleep131773-1kworker/1:0H23:32:421
24221214517,12sleep10-21swapper/123:13:331
8088214436,53sleep10-21swapper/122:23:111
8088214436,53sleep10-21swapper/122:23:111
2386214254,73sleep12406-21apt-get20:23:201
2386214254,73sleep12406-21apt-get20:23:201
7967214118,44sleep18043-21timerandwakeup20:38:591
1901821409,55sleep10-21swapper/119:33:301
636221390,7sleep00-21swapper/023:58:400
15834213944,40sleep115841-21munin-run22:48:101
27592213448,70sleep10-21swapper/123:23:381
24939213333,85sleep124913-21/usr/sbin/munin21:33:251
15023213312,106sleep215080-21cat00:28:102
15023213312,106sleep215080-21cat00:28:102
14605213118,96sleep214614-21/usr/sbin/munin22:43:262
16976212729,45sleep117002-21munin-run19:28:091
14976212632,43sleep11-21systemd00:28:091
14976212632,43sleep11-21systemd00:28:091
31177212410,97sleep131326-21latency_hist21:53:111
18772212412,23sleep018836-21memory21:13:330
16913212227,45sleep116399-21apt-get21:08:211
1959211922,72sleep02051-21sed20:23:110
1011821199,17sleep10-21swapper/122:28:321
4119211749,20sleep10-21swapper/120:28:291
6853211663,33sleep10-21swapper/120:38:101
1526021168,18sleep20-21swapper/200:28:222
12231211613,85sleep012239-21df00:18:270
12231211613,85sleep012239-21df00:18:270
10302211623,69sleep310338-21munin-run19:08:093
10302211623,69sleep310338-21munin-run19:08:093
7544211516,78sleep37602-21/usr/sbin/munin20:38:493
27901211515,84sleep228050-21date21:43:112
12530211521,79sleep312538-21needreboot00:18:343
12530211521,79sleep312538-21needreboot00:18:343
16450211417,43sleep110386-21cyclictest22:48:341
12366211315,77sleep212392-21cat19:13:172
12052211341,16sleep10-21swapper/120:53:241
595211026,40sleep10-21swapper/121:58:321
30587211013,78sleep330658-21/usr/sbin/munin23:33:373
19959211024,66sleep020109-21grep22:58:510
1157921105,38sleep10-21swapper/100:14:531
7235210911,82sleep07229-21apt-get00:03:250
2728321097,50sleep127380-21timerwakeupswit19:58:411
17491210912,81sleep017561-21swap21:09:010
1439521090,6sleep30-21swapper/322:43:103
1439521090,6sleep30-21swapper/322:43:103
15813210811,43sleep115873-21sshd00:28:371
12457210716,71sleep012521-21sendmail_mailqu20:53:350
6994210632,18sleep10-21swapper/100:03:101
1783021057,80sleep10-21swapper/122:53:311
23421210410,78sleep023375-21apt-get21:28:470
23421210410,78sleep023375-21apt-get21:28:470
17417210419,70sleep10-21swapper/100:33:421
10210210341,22sleep110440-21apt-config00:13:111
9146210236,17sleep10-21swapper/120:43:311
8963210229,47sleep28756-21/usr/sbin/munin00:08:282
3380210218,29sleep30-21swapper/322:08:113
5167210113,32sleep35234-21df22:13:243
31518210118,62sleep231483-21apt-get20:13:402
2447821014,87sleep00-21swapper/019:49:440
19178210114,67sleep018786-21/usr/sbin/munin19:33:330
3126121006,83sleep10-21swapper/123:37:061
2673921008,79sleep126749-21unin-run23:23:101
2300421005,49sleep10-21swapper/119:45:411
16026210016,63sleep316160-21/usr/sbin/munin00:28:393
1353621005,53sleep10-21swapper/120:58:161
1789929915,66sleep317605-21/usr/sbin/munin22:53:333
423229813,72sleep04349-21timerandwakeup22:08:400
235652986,38sleep10-21swapper/123:13:081
1259529820,57sleep212702-21timerandwakeup20:53:372
221082979,73sleep122202-21cut21:23:401
221082979,73sleep122202-21cut21:23:401
2015029710,72sleep220283-21latency_hist19:38:122
134242978,75sleep113427-21cron00:23:081
134242978,75sleep113427-21cron00:23:081
94522955,47sleep10-21swapper/122:26:461
94522955,47sleep10-21swapper/122:26:461
154692959,11sleep00-21swapper/000:28:280
293472945,79sleep00-21swapper/020:07:120
1018229413,64sleep010203-21latency_hist20:48:100
3260529314,13sleep00-21swapper/023:39:070
134452938,12sleep00-21swapper/022:38:360
134452938,12sleep00-21swapper/022:38:360
2707529210,61sleep20-21swapper/223:23:272
253429218,56sleep32025-21apt-get23:48:223
1893229213,19sleep10-21swapper/121:13:381
2401929116,22sleep30-21swapper/319:48:383
2675129020,54sleep326859-21latency_hist23:23:123
1419929016,22sleep10-21swapper/122:42:371
1419929016,22sleep10-21swapper/122:42:371
209412899,33sleep10-21swapper/119:38:391
901128823,45sleep18529-21apt-get19:03:191
901128823,45sleep18529-21apt-get19:03:191
3148628821,48sleep330993-21apt-get20:13:213
206428820,52sleep32142-21apt-key23:48:103
206428820,52sleep32142-21apt-key23:48:103
1728128814,61sleep317326-21processes00:33:403
118722880,14sleep311873-21latency_hist00:18:093
2102828713,61sleep021095-21latency23:03:310
83928530,42sleep1895-21tune2fs23:43:281
83928530,42sleep1895-21tune2fs23:43:281
3197128514,58sleep232057-21sed21:53:412
3134128418,47sleep231347-21cron23:38:082
1482728410,23sleep114958-21cat19:18:411
630928320,39sleep35652-21/usr/sbin/munin23:58:403
1928928325,37sleep20-21swapper/222:58:282
1784028315,55sleep317910-21switchtime19:28:403
836928224,48sleep10-21swapper/100:06:331
3177028214,48sleep00-21swapper/021:53:340
3053028219,40sleep230691-21tail21:48:402
581128115,30sleep00-21swapper/020:33:330
1878528127,27sleep30-21swapper/319:33:203
2701028018,32sleep30-21swapper/319:58:353
1169428025,31sleep20-21swapper/220:53:082
2815827920,43sleep10-21swapper/120:03:221
188827919,36sleep21902-21idleruntime-cro20:23:092
1184427919,39sleep211868-21modprobe00:18:082
971727826,32sleep00-21swapper/019:04:350
971727826,32sleep00-21swapper/019:04:350
544727819,38sleep35597-21latency_hist23:58:113
544727819,38sleep35597-21latency_hist23:58:113
292727812,21sleep30-21swapper/320:24:113
292727812,21sleep30-21swapper/320:24:113
277522785,62sleep10-21swapper/121:42:531
2520327824,31sleep20-21swapper/223:18:102
2005827818,24sleep00-21swapper/021:18:260
251182777,58sleep125212-21perl19:53:281
2196927722,30sleep00-21swapper/021:23:350
2196927722,30sleep00-21swapper/021:23:350
1844127616,29sleep30-21swapper/321:13:203
1071327621,42sleep10-21swapper/119:08:271
522627520,38sleep05228-21cron20:33:080
417327519,33sleep00-21swapper/023:53:280
417327519,33sleep00-21swapper/023:53:280
1708227511,48sleep216892-21/usr/sbin/munin21:08:432
1070327514,42sleep010712-21diskstats19:08:260
542827420,37sleep25446-21wc23:58:102
542827420,37sleep25446-21wc23:58:102
3108827422,33sleep231261-21latency_hist20:13:122
3108827422,33sleep231261-21latency_hist20:13:122
1550927417,36sleep215094-21/usr/sbin/munin21:03:342
1550927417,36sleep215094-21/usr/sbin/munin21:03:342
253827320,14sleep10-21swapper/123:48:251
186172737,13sleep00-21swapper/022:54:430
6102725,18sleep30-21swapper/321:58:343
51827211,45sleep20-21swapper/221:58:292
185027216,33sleep00-21swapper/022:03:110
1635127222,32sleep316440-21/usr/sbin/munin22:48:323
1225027210,30sleep20-21swapper/200:18:292
1225027210,30sleep20-21swapper/200:18:292
902227121,24sleep30-21swapper/322:23:413
902227121,24sleep30-21swapper/322:23:413
3186627130,22sleep20-21swapper/223:38:292
2232827115,33sleep322362-21/usr/sbin/munin19:43:333
206602716,13sleep20-21swapper/219:38:322
1723627118,40sleep10-21swapper/119:28:171
15962717,53sleep1363-21snmpd20:19:251
1497227115,34sleep014973-21cron00:28:080
1497227115,34sleep014973-21cron00:28:080
1489227121,30sleep00-21swapper/022:43:330
1062927115,18sleep20-21swapper/219:08:242
1099827026,24sleep30-21swapper/319:08:333
1010727022,22sleep00-21swapper/000:13:100
527726917,28sleep20-21swapper/220:33:102
2867026917,27sleep20-21swapper/221:43:352
1507826915,42sleep10-21swapper/121:03:131
1507826915,42sleep10-21swapper/121:03:131
1393426915,25sleep00-21swapper/019:18:110
1206726910,43sleep312103-21/usr/sbin/munin20:53:263
2288026818,38sleep222965-21switchtime23:08:372
2867126716,21sleep20-21swapper/220:03:392
177052678,14sleep20-21swapper/222:53:272
888126618,29sleep20-21swapper/220:43:242
760726619,30sleep37654-21users22:18:413
760726619,30sleep37654-21users22:18:413
271226622,25sleep00-21swapper/023:48:480
2687826622,24sleep00-21swapper/019:58:320
2660026616,21sleep00-21swapper/021:38:280
1981726615,34sleep219919-21sed21:18:112
578326517,25sleep30-21swapper/320:33:323
294882656,49sleep00-21swapper/021:47:160
1791526521,24sleep00-21swapper/019:28:410
1419326518,24sleep30-21swapper/300:23:393
1419326518,24sleep30-21swapper/300:23:393
138892656,11sleep20-21swapper/222:38:422
138892656,11sleep20-21swapper/222:38:422
2630626420,19sleep30-21swapper/321:38:113
2530026421,21sleep00-21swapper/019:53:320
1721526415,32sleep017265-21sshd00:33:370
1616726419,28sleep00-21swapper/022:48:290
1030726419,19sleep20-21swapper/222:28:372
878226313,14sleep00-21swapper/022:23:380
878226313,14sleep00-21swapper/022:23:380
214626317,29sleep02358-21latency_hist23:48:110
214626317,29sleep02358-21latency_hist23:48:110
2068726323,17sleep30-21swapper/321:18:413
1456526318,21sleep30-21swapper/319:18:343
3034326214,27sleep00-21swapper/023:33:300
2860926220,21sleep30-21swapper/320:03:363
27442629,23sleep20-21swapper/220:24:062
27442629,23sleep20-21swapper/220:24:062
2337926217,28sleep322879-21apt-get21:28:223
2337926217,28sleep322879-21apt-get21:28:223
2057926213,24sleep00-21swapper/019:38:300
1579126217,23sleep20-21swapper/219:23:262
1520326218,25sleep00-21swapper/021:03:280
1520326218,25sleep00-21swapper/021:03:280
920226125,19sleep20-21swapper/219:04:252
920226125,19sleep20-21swapper/219:04:252
83526112,35sleep3886-21cat20:18:303
651626121,23sleep20-21swapper/222:18:112
3096826117,28sleep031007-21apt-get20:13:100
3096826117,28sleep031007-21apt-get20:13:100
1703026112,33sleep017119-21ls19:28:120
1190826122,18sleep30-21swapper/322:33:383
1190826122,18sleep30-21swapper/322:33:383
410126017,25sleep24163-21/usr/sbin/munin22:08:382
300542609,34sleep030112-21/usr/sbin/munin21:48:300
2805126021,17sleep30-21swapper/321:43:123
2502226016,21sleep00-21swapper/021:33:280
243862606,11sleep20-21swapper/221:29:462
243862606,11sleep20-21swapper/221:29:462
1313026015,26sleep313198-21/usr/sbin/munin19:13:383
105392608,37sleep00-21swapper/000:13:140
643625913,32sleep06504-21modprobe22:18:090
423325916,25sleep34307-21sed20:28:313
296825923,22sleep00-21swapper/020:24:120
296825923,22sleep00-21swapper/020:24:120
2862125921,22sleep20-21swapper/223:28:272
2285725919,21sleep30-21swapper/321:28:103
2285725919,21sleep30-21swapper/321:28:103
2061125919,18sleep20-21swapper/221:18:402
193632597,11sleep30-21swapper/322:58:293
1651525914,26sleep30-21swapper/321:08:113
1651525914,26sleep30-21swapper/321:08:113
1631625912,27sleep20-21swapper/222:48:312
385425818,26sleep33888-21/usr/sbin/munin22:08:313
2228125814,26sleep022300-21apt-get23:08:190
1004325812,30sleep310105-21apt-config00:13:093
955825712,22sleep20-21swapper/222:28:082
955825712,22sleep20-21swapper/222:28:082
761125713,25sleep27647-21ntpq00:03:352
3136825717,24sleep331387-21apt-get21:53:213
273892578,13sleep00-21swapper/023:23:330
2112725716,18sleep20-21swapper/223:03:332
1463325719,19sleep20-21swapper/219:18:382
1176925717,26sleep211798-21/usr/sbin/munin22:33:332
1176925717,26sleep211798-21/usr/sbin/munin22:33:332
116222575,10sleep30-21swapper/320:52:133
1116625718,17sleep00-21swapper/022:33:110
892625619,23sleep38952-21df20:43:253
694325618,23sleep27138-21apt-config20:38:112
612825616,24sleep26130-21ntp_states23:58:352
206925615,25sleep02063-21apt-get22:03:230
156225613,23sleep21654-21users23:43:412
156225613,23sleep21654-21users23:43:412
1493125614,18sleep00-21swapper/021:03:120
2587325514,24sleep225892-21ntp_states23:18:352
2521325518,21sleep20-21swapper/219:53:292
2442925516,23sleep324518-21sed23:13:393
2084925515,24sleep320363-21/usr/sbin/munin19:38:373
1099225515,27sleep011073-21switchtime20:48:380
764925415,23sleep37687-21ntp_states00:03:363
683825421,15sleep20-21swapper/222:18:272
683825421,15sleep20-21swapper/222:18:272
734825312,24sleep27320-21apt-get20:38:352
2361625313,16sleep30-21swapper/323:13:103
2351525314,23sleep223554-21df19:48:252
1403025313,20sleep30-21swapper/320:58:393
2970425212,22sleep329700-21apt-get20:08:203
2970425212,22sleep329700-21apt-get20:08:203
2555625211,23sleep025646-21perl23:18:290
243725218,17sleep20-21swapper/222:03:342
1775325213,15sleep20-21swapper/219:28:392
1749525216,22sleep217636-21unixbench_multi00:33:432
1453125212,25sleep314526-21apt-get22:43:233
1424725213,15sleep00-21swapper/019:18:270
675125113,22sleep06746-21apt-get22:18:210
675125113,22sleep06746-21apt-get22:18:210
3210025111,23sleep332111-21ntp_states23:38:343
2984625114,22sleep329913-21df21:48:253
2678225113,17sleep30-21swapper/321:38:313
1021725012,18sleep30-21swapper/322:28:343
2853124912,22sleep328529-21apt-get23:28:203
166882498,32sleep30-21swapper/319:24:283
2636024810,17sleep30-21swapper/319:58:113
2236424812,22sleep022386-21ntp_states19:43:340
1276324814,17sleep00-21swapper/022:38:110
1276324814,17sleep00-21swapper/022:38:110
2739624712,19sleep20-21swapper/219:58:422
2735424718,16sleep30-21swapper/323:23:323
73262469,14sleep00-21swapper/020:38:210
315902448,18sleep00-21swapper/020:13:460
2983624411,17sleep00-21swapper/023:33:100
135682427,17sleep20-21swapper/220:58:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional