You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-12 - 20:01

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue May 12, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
15792210937,1048sleep015831-21ntpq08:49:580
29324253011,506sleep229349-21ntpq07:50:002
11506253016,500sleep311519-21irqstats10:19:543
11506253016,500sleep311519-21irqstats10:19:543
1850324156,399sleep0363-21snmpd08:59:440
24232238216,350sleep124234-21/usr/sbin/munin07:34:451
30354233327,241sleep028829-1kworker/0:1H09:35:210
30354233327,241sleep028829-1kworker/0:1H09:35:210
446823030,8sleep00-21swapper/008:14:440
136322927,200sleep00-21swapper/008:04:440
11220228511,263sleep011233-21cat10:19:430
11220228511,263sleep011233-21cat10:19:430
175242274105,65sleep017599-21switchtime08:55:030
15850225793,26sleep00-21swapper/007:09:360
9291225327,214sleep0363-21snmpd10:13:290
25621225248,61sleep00-21swapper/011:04:570
8781225016,221sleep38820-21sort11:54:553
15655222018,181sleep115741-21latency_hist10:34:341
15655222018,181sleep115741-21latency_hist10:34:341
11817222013,193sleep011885-21fschecks_time12:04:540
32744221829,37sleep0315-21munin-plugin-st11:29:320
32744221829,37sleep0315-21munin-plugin-st11:29:320
6049221616,154sleep06059-21acpi08:19:420
28932221511,99sleep00-21swapper/007:49:500
10142221539,67sleep010208-21sendmail_mailqu10:15:030
10142221539,67sleep010208-21sendmail_mailqu10:15:030
3526221424,150sleep03551-21apt-key11:39:340
20913221250,140sleep020930-21latency_hist07:24:340
393220812,179sleep0541-21kworker/u8:211:29:370
393220812,179sleep0541-21kworker/u8:211:29:370
30680220623,161sleep230830-21unixbench_singl11:20:062
19598220690,21sleep00-21swapper/010:45:030
23173219512,167sleep023207-21ps07:30:030
1008421887,168sleep00-21swapper/008:30:080
1008421887,168sleep00-21swapper/008:30:080
13545218011,149sleep013563-21grep12:09:540
13545218011,149sleep013563-21grep12:09:540
6655217912,60sleep00-21swapper/010:04:530
21260217026,123sleep121402-21unixbench_multi10:50:051
31696216929,127sleep00-21swapper/007:59:070
1405121658,43sleep00-21swapper/008:44:540
4529216110,113sleep04621-21switchtime11:40:070
4529216110,113sleep04621-21switchtime11:40:070
21730216021,124sleep00-21swapper/009:09:530
20022215922,112sleep00-21swapper/012:30:010
2848421589,113sleep028593-21ls09:30:040
2608721509,38sleep00-21swapper/009:24:350
1644214912,52sleep00-21swapper/009:49:350
830821459,101sleep08342-21idleruntime-cro11:54:320
1785321459,97sleep00-21swapper/010:39:580
16712214421,41sleep00-21swapper/012:19:550
2706421435,28sleep00-21swapper/007:44:130
15101214326,94sleep315122-21modprobe08:49:333
2103321415,29sleep00-21swapper/012:34:370
2103321415,29sleep00-21swapper/012:34:370
8971214016,103sleep00-21swapper/008:29:350
11461214021,98sleep311596-21grep08:35:073
11461214021,98sleep311596-21grep08:35:073
2256213112,18sleep20-21swapper/209:49:572
3128921305,42sleep00-21swapper/009:39:490
21619213018,21sleep20-21swapper/212:35:032
21619213018,21sleep20-21swapper/212:35:032
21572212821,89sleep121617-21ntp_states07:25:041
2102321287,105sleep00-21swapper/010:49:590
19148212619,96sleep00-21swapper/007:17:400
53821255,19sleep00-21swapper/008:00:320
15373212582,24sleep00-21swapper/012:15:030
21341212113,91sleep021344-21cron09:09:330
15739212129,67sleep215772-21/usr/sbin/munin07:09:342
30483211836,46sleep00-21swapper/011:20:040
17374211825,70sleep217497-21latency_hist07:14:352
17374211825,70sleep217497-21latency_hist07:14:352
1700221185,12sleep00-21swapper/007:10:250
1700221185,12sleep00-21swapper/007:10:250
9636211720,78sleep29638-21/usr/sbin/munin10:14:442
9636211720,78sleep29638-21/usr/sbin/munin10:14:442
23967211722,78sleep123981-21apt07:34:321
19846211623,29sleep30-21swapper/309:04:353
11100211627,50sleep00-21swapper/008:35:000
11100211627,50sleep00-21swapper/008:35:000
2735721145,96sleep00-21swapper/007:44:450
2735721145,96sleep00-21swapper/007:44:450
619721130,11sleep10-21swapper/108:19:501
2498211314,84sleep12535-21ntp_kernel_err11:34:591
27583211124,70sleep127596-21apt09:29:331
2460211114,83sleep32461-21sshd08:07:093
12585211121,24sleep30-21swapper/308:39:563
1063521118,90sleep0363-21snmpd12:00:010
387321088,89sleep0363-21snmpd08:10:210
26094210715,77sleep326114-21kernelversion07:39:563
23262210721,45sleep00-21swapper/009:14:500
343321068,39sleep00-21swapper/009:54:480
6144210514,77sleep26152-21df08:19:482
24505210522,45sleep00-21swapper/007:34:560
2229121055,89sleep00-21swapper/007:27:570
215521059,62sleep02210-21latency09:49:550
15986210519,45sleep00-21swapper/010:34:530
593321047,46sleep00-21swapper/011:44:590
593321047,46sleep00-21swapper/011:44:590
16154210425,57sleep116235-21fschecks_time07:09:531
16154210425,57sleep116235-21fschecks_time07:09:531
29957210316,70sleep330049-21/usr/sbin/munin09:35:123
29957210316,70sleep330049-21/usr/sbin/munin09:35:123
28595210322,60sleep128734-21sed09:30:051
3917210228,48sleep23412-21apt-get11:39:442
3917210228,48sleep23412-21apt-get11:39:442
23700210211,49sleep00-21swapper/010:59:490
23700210211,49sleep00-21swapper/010:59:490
15799210239,30sleep30-21swapper/307:09:353
9780210116,69sleep39866-21fschecks_time10:14:533
9780210116,69sleep39866-21fschecks_time10:14:533
15489210122,57sleep315598-21cut08:49:533
15120210145,42sleep015240-21unixbench_singl10:30:050
15120210145,42sleep015240-21unixbench_singl10:30:050
4087210010,75sleep24089-21sshd08:13:102
18251210011,76sleep218284-21/usr/sbin/munin08:59:332
1316621008,15sleep00-21swapper/010:24:560
284302997,76sleep00-21swapper/011:14:510
284302997,76sleep00-21swapper/011:14:510
1724129915,64sleep20-21swapper/210:39:362
135862988,78sleep013602-21munin-run08:44:330
194329710,71sleep22011-21switchtime08:05:062
91992969,14sleep10-21swapper/110:12:171
76382967,12sleep20-21swapper/210:07:092
1760229615,64sleep117722-21/usr/sbin/munin08:55:041
1680129620,57sleep216850-21awk12:19:572
1132729610,75sleep10-21swapper/112:03:051
1560429522,50sleep315630-21munin-plugin-st10:34:323
1560429522,50sleep315630-21munin-plugin-st10:34:323
50952947,46sleep00-21swapper/009:59:520
312152940,9sleep30-21swapper/309:39:433
305812949,12sleep20-21swapper/209:35:232
305812949,12sleep20-21swapper/209:35:232
82152939,11sleep10-21swapper/111:53:201
280022938,12sleep30-21swapper/311:13:453
2211529319,59sleep222165-21processes09:10:032
2316629215,56sleep223167-21/usr/sbin/munin09:14:442
1174729214,63sleep211823-21/usr/sbin/munin10:20:012
1174729214,63sleep211823-21/usr/sbin/munin10:20:012
1760629112,67sleep217636-21timerandwakeup07:14:392
1928129019,24sleep20-21swapper/212:29:342
1665929020,46sleep216688-21awk08:54:332
188028916,52sleep11942-21/usr/sbin/munin08:05:051
510028818,53sleep15152-21cut09:59:531
3205328828,33sleep20-21swapper/207:59:492
1951828819,41sleep219512-21apt-get12:29:472
312082870,9sleep10-21swapper/111:24:321
75672869,10sleep30-21swapper/310:06:133
2845028632,24sleep20-21swapper/211:14:532
2845028632,24sleep20-21swapper/211:14:532
254392869,67sleep20-21swapper/207:38:372
2308828630,10sleep00-21swapper/010:55:450
2308828630,10sleep00-21swapper/010:55:450
1256428626,40sleep212583-21sed08:39:552
324572855,68sleep20-21swapper/211:25:512
324572855,68sleep20-21swapper/211:25:512
33972846,17sleep20-21swapper/209:54:372
3205828419,40sleep30-21swapper/311:25:033
3205828419,40sleep30-21swapper/311:25:033
314228427,39sleep13169-21grep09:54:321
203192840,7sleep30-21swapper/312:30:033
1336528419,49sleep113406-21fschecks_count12:09:511
1336528419,49sleep113406-21fschecks_count12:09:511
118852848,64sleep10-21swapper/108:38:251
118852848,64sleep10-21swapper/108:38:251
835828320,46sleep28448-21latency_hist11:54:342
3241428325,25sleep30-21swapper/307:59:583
197992830,8sleep20-21swapper/210:45:042
394628211,54sleep10-21swapper/111:39:471
394628211,54sleep10-21swapper/111:39:471
2063428225,38sleep220704-21grep09:05:032
851728115,19sleep20-21swapper/210:10:002
2970028018,38sleep129815-21unixbench_singl07:50:071
2633228043,23sleep026319-21apt-get09:24:510
205732807,11sleep10-21swapper/107:20:441
1909828024,12sleep00-21swapper/012:27:510
1867628014,52sleep318817-21unixbench_singl12:25:033
1861728020,47sleep118629-21fschecks_count08:59:531
1161628017,32sleep10-21swapper/110:19:561
1161628017,32sleep10-21swapper/110:19:561
96092795,42sleep00-21swapper/011:56:220
241722795,63sleep10-21swapper/109:16:001
139642799,59sleep30-21swapper/310:28:223
1205527925,29sleep20-21swapper/208:39:352
1205527925,29sleep20-21swapper/208:39:352
515327814,40sleep35278-21timerwakeupswit08:15:053
108892789,11sleep10-21swapper/110:19:031
108892789,11sleep10-21swapper/110:19:031
2688327726,29sleep30-21swapper/309:25:063
2653827716,31sleep20-21swapper/209:24:572
158802775,59sleep20-21swapper/212:16:382
322732767,57sleep10-21swapper/109:41:401
2777227618,10sleep00-21swapper/011:10:470
2187827622,37sleep322023-21sed07:25:103
2167627614,28sleep30-21swapper/312:35:043
2167627614,28sleep30-21swapper/312:35:043
1492327622,35sleep20-21swapper/210:30:032
1492327622,35sleep20-21swapper/210:30:032
3041927515,42sleep330481-21/usr/sbin/munin11:20:033
32527416,16sleep30-21swapper/309:44:493
31182747,58sleep20-21swapper/211:36:052
2824427410,20sleep20-21swapper/209:29:582
2477827422,25sleep20-21swapper/211:02:092
2477827422,25sleep20-21swapper/211:02:092
176352744,10sleep30-21swapper/312:23:583
1392727416,44sleep214006-21fschecks_time08:44:522
319227325,28sleep30-21swapper/309:54:343
3006927317,32sleep130112-21ntpq09:35:141
3006927317,32sleep130112-21ntpq09:35:141
2970227316,33sleep129833-21latency_hist11:19:361
2970227316,33sleep129833-21latency_hist11:19:361
2714327324,28sleep20-21swapper/207:44:352
2302027320,30sleep222099-21/usr/sbin/munin10:55:062
2302027320,30sleep222099-21/usr/sbin/munin10:55:062
128227315,37sleep3552-21/usr/sbin/munin11:30:043
128227315,37sleep3552-21/usr/sbin/munin11:30:043
586627216,42sleep25893-21awk11:44:572
586627216,42sleep25893-21awk11:44:572
2658327225,27sleep10-21swapper/111:09:351
208162724,57sleep30-21swapper/307:23:523
1873927222,31sleep218773-21awk10:44:322
749527112,14sleep30-21swapper/311:49:593
3194727121,30sleep10-21swapper/111:25:011
3194727121,30sleep10-21swapper/111:25:011
239827112,42sleep32476-21sed11:34:573
238392716,56sleep30-21swapper/307:32:573
396627026,25sleep30-21swapper/309:55:033
722226915,38sleep17341-21timerwakeupswit10:05:061
2097926912,41sleep120988-21/usr/sbin/munin09:05:061
1902726913,27sleep10-21swapper/110:44:481
2348526819,27sleep30-21swapper/310:59:353
2348526819,27sleep30-21swapper/310:59:353
232932685,51sleep10-21swapper/110:58:241
232932685,51sleep10-21swapper/110:58:241
1922226722,21sleep20-21swapper/209:00:062
1828526715,30sleep30-21swapper/308:59:343
1519826720,24sleep10-21swapper/107:05:051
90592667,12sleep20-21swapper/211:55:022
2795426620,27sleep328032-21ls07:45:043
2795426620,27sleep328032-21ls07:45:043
1599426620,25sleep30-21swapper/312:18:093
101526620,31sleep21153-21unixbench_multi09:45:042
762526413,26sleep27618-21apt-get08:24:492
3054826416,22sleep30-21swapper/307:54:503
2295226414,37sleep123025-21cat09:14:351
1794426418,19sleep30-21swapper/310:40:013
1219726415,29sleep112257-21/usr/sbin/munin12:05:031
827526312,27sleep30-21swapper/308:25:063
667726323,19sleep30-21swapper/308:20:013
3187526321,18sleep10-21swapper/107:59:361
2772126312,20sleep20-21swapper/211:10:082
1630326318,26sleep30-21swapper/310:35:003
430226222,19sleep20-21swapper/208:14:352
2603426220,19sleep20-21swapper/209:24:332
2108526217,23sleep30-21swapper/310:50:023
2033826222,21sleep30-21swapper/310:49:343
508426114,20sleep30-21swapper/309:59:513
499226112,38sleep30-21swapper/311:41:433
499226112,38sleep30-21swapper/311:41:433
456126112,33sleep14567-21df_inode08:14:501
424726112,33sleep14301-21cat08:14:341
2569526113,33sleep325696-21sshd09:20:103
2016926115,30sleep320181-21fschecks_count09:04:513
1610426117,28sleep116129-21munin-run12:19:321
1444526116,26sleep114531-21timerandwakeup08:45:031
1282326014,19sleep10-21swapper/108:40:021
980625925,17sleep20-21swapper/208:30:022
980625925,17sleep20-21swapper/208:30:022
556925916,20sleep30-21swapper/311:44:503
556925916,20sleep30-21swapper/311:44:503
2756525921,20sleep20-21swapper/207:44:552
2756525921,20sleep20-21swapper/207:44:552
1811125915,22sleep30-21swapper/307:14:523
2731225819,25sleep127405-21cut11:10:031
1855925812,29sleep10-21swapper/107:15:021
1072725821,17sleep20-21swapper/212:00:032
3150725716,26sleep231543-21latency09:39:562
2278925714,19sleep20-21swapper/207:29:532
1679025715,23sleep10-21swapper/112:19:561
811825611,28sleep38111-21apt-get10:09:493
548125614,18sleep20-21swapper/210:00:022
389425621,15sleep10-21swapper/109:55:011
2919125620,19sleep30-21swapper/307:49:573
2629525620,20sleep126332-21ps07:40:021
2143425626,16sleep20-21swapper/207:25:012
1081125618,23sleep310933-21sed12:00:043
577125517,21sleep15865-21memory11:44:561
577125517,21sleep15865-21memory11:44:561
57525514,23sleep1570-21apt-get11:29:461
57525514,23sleep1570-21apt-get11:29:461
32325513,25sleep1317-21apt-get09:44:471
292502557,11sleep30-21swapper/311:15:063
292502557,11sleep30-21swapper/311:15:063
2809425514,22sleep328143-21irqstats09:29:553
1692225513,24sleep316969-21users07:10:083
1692225513,24sleep316969-21users07:10:083
886825418,18sleep10-21swapper/111:54:571
2525325412,25sleep225249-21apt-get11:04:472
2521025415,23sleep125234-21ntp_states09:20:001
1821525415,24sleep118238-21irqstats12:24:531
1410925424,14sleep20-21swapper/210:29:342
3065025321,19sleep20-21swapper/207:54:532
2869225312,16sleep10-21swapper/107:49:351
2869225312,16sleep10-21swapper/107:49:351
2501425317,16sleep10-21swapper/111:04:341
2501425317,16sleep10-21swapper/111:04:341
2155225320,18sleep10-21swapper/112:35:021
2155225320,18sleep10-21swapper/112:35:021
1982325313,24sleep219872-21if_eth007:19:562
1772025313,24sleep117752-21irqstats10:39:551
952925216,23sleep39543-21grep08:29:553
952925216,23sleep39543-21grep08:29:553
291625218,18sleep30-21swapper/308:09:443
2412925214,24sleep324153-21ntp_states11:00:003
2412925214,24sleep324153-21ntp_states11:00:003
2187525212,25sleep221900-21sed10:54:342
1590625212,23sleep115899-21apt-get10:34:501
1172925212,23sleep211725-21apt-get12:04:472
959125112,19sleep10-21swapper/108:29:571
959125112,19sleep10-21swapper/108:29:571
2474125115,17sleep30-21swapper/307:35:033
1959725110,16sleep10-21swapper/112:29:501
770225017,17sleep10-21swapper/108:24:521
314152507,31sleep10-21swapper/107:55:281
1583325013,22sleep115850-21ntp_states08:49:591
2533824913,22sleep325358-21fschecks_count11:04:523
2317524912,22sleep323186-21apt-get09:14:453
1453824912,14sleep20-21swapper/212:14:322
1453824912,14sleep20-21swapper/212:14:322
1378324911,24sleep313874-21mailstats12:10:013
1378324911,24sleep313874-21mailstats12:10:013
1329224911,22sleep113371-21sendmail_mailtr10:25:011
118524814,20sleep21281-21timerandwakeup11:30:032
118524814,20sleep21281-21timerandwakeup11:30:032
2204824317,14sleep30-21swapper/309:10:003
117302418,16sleep30-21swapper/312:04:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional