You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-02 - 04:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Aug 02, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20274219769,1954sleep315724-21kworker/3:200:30:383
32137210339,1007sleep326563-1kworker/3:0H21:42:483
3143296320,921sleep33267-21sshd23:37:393
22427283417,800sleep321489-21/usr/sbin/munin22:53:113
46862597557,26sleep04688-21sshd23:40:380
25412258316,439sleep125428-21ls21:22:361
250632420250,147sleep125156-21fschecks_time19:38:021
294192353157,179sleep129458-21fschecks_count23:17:591
749822819,255sleep0195-21jbd2/mmcblk2p2-22:07:520
749822819,255sleep0195-21jbd2/mmcblk2p2-22:07:520
15251227723,233sleep015354-21unixbench_multi20:48:120
9574227051,198sleep10-21swapper/123:57:391
9574227051,198sleep10-21swapper/123:57:391
25462227018,237sleep325467-21latency_hist21:22:383
30949224753,167sleep131089-21latency_hist19:57:391
20477223614,206sleep320518-21idleruntime-cro21:07:373
2303223238,173sleep12343-21proc_pri23:34:061
15052221124,167sleep115125-21sshd22:32:391
11988221014,181sleep112075-21timerandwakeup00:03:121
11988221014,181sleep112075-21timerandwakeup00:03:121
24644219914,169sleep124686-21awk19:37:371
18176219817,160sleep118303-21latency_hist22:42:401
18176219817,160sleep118303-21latency_hist22:42:401
14996219218,21sleep10-21swapper/119:08:021
2854621869,160sleep128041-21/usr/sbin/munin19:48:071
23078218317,149sleep123161-21mailstats21:13:081
15399217913,151sleep115497-21fschecks_time22:32:591
12352169132,17sleep10-21swapper/121:47:471
19618216816,138sleep119658-21grep21:03:031
16072216619,130sleep016066-21apt-get00:17:520
7006215715,129sleep17091-21cut20:23:071
2877721578,55sleep10-21swapper/121:32:501
11805215618,114sleep10-21swapper/120:38:041
31682215310,127sleep131772-21apt-config21:42:381
26165215317,27sleep10-21swapper/121:23:051
19938215116,31sleep119948-21acpi22:47:481
2510421498,51sleep10-21swapper/123:03:061
2510421498,51sleep10-21swapper/123:03:061
18016214915,114sleep118104-21sshd19:17:391
9452214815,35sleep19538-21wc20:32:391
29695214824,99sleep229711-21sort19:52:562
21097214742,90sleep121144-21ntp_kernel_pll_00:33:021
16255214612,109sleep10-21swapper/120:52:581
35962145104,18sleep10-21swapper/120:13:011
35962145104,18sleep10-21swapper/120:13:011
29090214519,109sleep129206-21latency_hist23:17:391
3133621448,121sleep00-21swapper/021:38:460
3318214213,22sleep10-21swapper/121:53:081
1021214112,66sleep11112-21sort21:47:391
6750213710,61sleep10-21swapper/123:47:541
7987213413,104sleep28057-21sed23:52:392
22354213014,100sleep122516-21latency_hist21:12:391
10838213019,93sleep10-21swapper/122:17:581
32236212911,21sleep132390-21modprobe23:27:401
2435621299,107sleep124385-21munin-run23:02:371
2435621299,107sleep124385-21munin-run23:02:371
3275212312,95sleep03315-21ps21:53:070
18148212312,44sleep117640-21/usr/sbin/munin20:58:061
18148212312,44sleep117640-21/usr/sbin/munin20:58:061
1079721239,38sleep10-21swapper/120:34:291
30057212025,76sleep030085-21munin-run21:37:370
18878211923,72sleep018891-21cron00:27:370
9846211820,80sleep29344-21apt-get20:32:482
26510211810,14sleep10-21swapper/119:42:501
20026211810,96sleep116121-21munin-node19:22:481
13059211810,45sleep113068-21diskstats20:42:571
3087921177,16sleep10-21swapper/123:22:511
19333211713,16sleep00-21swapper/000:27:570
8522211615,79sleep38529-21/usr/sbin/munin20:28:053
18982211610,92sleep219007-21sed21:02:392
18982211610,92sleep219007-21sed21:02:392
555211510,34sleep10-21swapper/120:03:001
20034211516,79sleep320040-21/usr/sbin/munin22:47:543
32135211318,76sleep231650-21apt-get21:42:472
2577211322,68sleep02724-21grep20:08:120
1653421139,41sleep116568-21ps00:18:051
13222211322,75sleep313251-21fschecks_count19:02:563
4652211250,34sleep04675-21ls20:17:370
4652211250,34sleep04675-21ls20:17:370
1841221125,13sleep00-21swapper/019:17:420
1218621127,15sleep10-21swapper/122:22:541
580521109,29sleep10-21swapper/123:43:101
31188211011,34sleep10-21swapper/119:57:521
19585211011,38sleep10-21swapper/100:28:021
487521088,29sleep10-21swapper/123:42:371
466221089,14sleep10-21swapper/121:58:001
2899210821,66sleep32929-21fschecks_count21:52:583
2724821087,38sleep10-21swapper/121:27:491
2114521080,13sleep20-21swapper/200:33:022
16841210826,57sleep216860-21apt-get22:37:482
16841210826,57sleep216860-21apt-get22:37:482
1768210715,17sleep00-21swapper/023:32:550
30350210616,74sleep230355-21sort21:37:472
1376921066,32sleep10-21swapper/100:08:081
1376921066,32sleep10-21swapper/100:08:081
30635210411,76sleep030679-21date23:22:380
2272721048,29sleep10-21swapper/122:57:001
2146021045,28sleep10-21swapper/119:27:401
1561521046,30sleep10-21swapper/120:50:151
1550521047,29sleep10-21swapper/100:14:341
869121035,29sleep10-21swapper/122:11:481
869121035,29sleep10-21swapper/122:11:481
1792621035,14sleep00-21swapper/022:40:040
1792621035,14sleep00-21swapper/022:40:040
1085121027,28sleep10-21swapper/123:59:361
1085121027,28sleep10-21swapper/123:59:361
604121015,27sleep10-21swapper/120:20:121
1014121015,29sleep10-21swapper/122:15:231
708521007,29sleep10-21swapper/122:06:031
708521007,29sleep10-21swapper/122:06:031
2659121005,29sleep10-21swapper/123:08:021
2659121005,29sleep10-21swapper/123:08:021
1855821006,28sleep10-21swapper/100:23:301
2403229914,17sleep20-21swapper/219:33:102
185072997,78sleep10-21swapper/119:17:551
18442988,28sleep10-21swapper/120:07:571
1943829714,67sleep219497-21latency00:27:592
1603129615,64sleep216142-21latency_hist19:12:392
1340629623,59sleep213447-21latency19:02:592
194342955,28sleep10-21swapper/122:44:331
194342955,28sleep10-21swapper/122:44:331
1770729515,66sleep317763-21tune2fs00:22:553
3074329319,52sleep330766-21netstat21:38:003
1847429323,46sleep317964-21apt-get19:17:493
1423829320,50sleep014326-21grep00:12:390
1423829320,50sleep014326-21grep00:12:390
452929215,22sleep20-21swapper/223:38:392
227302914,75sleep00-21swapper/019:30:580
1270629013,61sleep012856-21latency_hist00:07:390
1270629013,61sleep012856-21latency_hist00:07:390
2383728819,51sleep223862-21modprobe21:17:372
309602879,61sleep00-21swapper/023:22:550
788228614,18sleep20-21swapper/220:27:402
1977628616,17sleep00-21swapper/022:47:390
1900828614,56sleep319160-21cat21:02:403
1332828529,36sleep212901-21/usr/sbin/munin00:08:012
1332828529,36sleep212901-21/usr/sbin/munin00:08:012
47228413,57sleep0552-21readlink23:28:070
2627628422,41sleep326350-21cat21:23:073
2214628416,45sleep022274-21timerwakeupswit22:53:080
2982128219,25sleep30-21swapper/323:18:093
1538128218,48sleep015398-21uniq22:32:580
1893228113,49sleep018971-21modprobe21:02:370
1893228113,49sleep018971-21modprobe21:02:370
1345228115,50sleep013512-21/usr/sbin/munin20:43:080
317932800,9sleep212-21rcu_sched19:58:082
1334728021,24sleep10-21swapper/119:02:581
485027918,43sleep24851-21ntp_states21:58:052
304192797,49sleep30-21swapper/319:53:093
2789227918,28sleep20-21swapper/221:28:062
1358927920,45sleep213672-21wc20:43:102
112302790,8sleep20-21swapper/222:18:072
2442627818,23sleep00-21swapper/021:17:590
815727717,38sleep08244-21fschecks_time20:27:580
729127615,30sleep00-21swapper/020:23:100
2579027616,38sleep025871-21grep21:22:560
211027617,39sleep32169-21latency20:08:033
3165727519,24sleep00-21swapper/019:58:070
2005327520,27sleep20-21swapper/219:22:532
709227415,25sleep20-21swapper/220:23:082
66732737,49sleep00-21swapper/023:47:490
3265527319,25sleep00-21swapper/021:43:220
3013527320,32sleep230224-21latency_hist21:37:392
2517927313,46sleep225181-21sshd21:19:392
1735927318,24sleep00-21swapper/019:13:100
1693927315,39sleep316959-21kernelversion19:13:013
963427217,27sleep20-21swapper/223:57:402
963427217,27sleep20-21swapper/223:57:402
65127216,43sleep2668-21netstat20:03:012
2355527221,33sleep00-21swapper/019:33:000
1022527213,43sleep310239-21/usr/sbin/munin20:34:193
562827116,40sleep35758-21sed20:18:083
3011527117,25sleep00-21swapper/019:53:060
2687227113,30sleep30-21swapper/323:08:083
2687227113,30sleep30-21swapper/323:08:083
223727114,37sleep22271-21ntp_kernel_pll_23:34:032
1635827121,35sleep30-21swapper/320:53:003
1273927120,22sleep30-21swapper/320:42:383
2648627013,39sleep326504-21apt19:42:483
1617626918,27sleep00-21swapper/020:52:570
2347726822,18sleep30-21swapper/319:32:593
1886726816,26sleep20-21swapper/219:19:132
1515026820,28sleep30-21swapper/320:48:113
145026811,42sleep31511-21if_eth021:47:573
1253326817,20sleep00-21swapper/022:23:010
972626715,33sleep09883-21unixbench_multi22:13:080
93762667,12sleep00-21swapper/023:56:120
93762667,12sleep00-21swapper/023:56:120
2801426616,30sleep30-21swapper/323:13:013
135626617,33sleep21447-21cut21:47:562
609326525,25sleep20-21swapper/222:02:572
609326525,25sleep20-21swapper/222:02:572
2782326524,25sleep227936-21sshd19:47:392
1352126523,21sleep20-21swapper/222:27:392
948226415,29sleep39545-40mandb23:57:373
948226415,29sleep39545-40mandb23:57:373
493226418,27sleep35086-21latency_hist23:42:393
491826413,20sleep00-21swapper/021:58:070
2656426419,24sleep00-21swapper/023:08:010
2656426419,24sleep00-21swapper/023:08:010
2542526412,37sleep025431-21ntp_states19:38:090
2355226416,26sleep00-21swapper/022:58:040
2355226416,26sleep00-21swapper/022:58:040
2070226420,22sleep20-21swapper/221:07:402
194232648,10sleep00-21swapper/022:44:240
194232648,10sleep00-21swapper/022:44:240
1841626413,22sleep20-21swapper/222:42:532
1841626413,22sleep20-21swapper/222:42:532
1487326415,30sleep30-21swapper/319:08:003
1269026419,24sleep20-21swapper/220:42:372
2338326210,37sleep023474-21users21:13:110
2213226215,24sleep20-21swapper/219:28:052
2142626215,26sleep30-21swapper/321:08:093
1723626219,24sleep30-21swapper/322:38:023
1723626219,24sleep30-21swapper/322:38:023
130042629,22sleep30-21swapper/300:07:543
130042629,22sleep30-21swapper/300:07:543
127026218,27sleep31308-21munin-run23:32:383
2205126114,29sleep322058-21/usr/sbin/munin19:28:023
1131126118,23sleep30-21swapper/322:18:083
106232615,14sleep00-21swapper/022:17:490
807026017,29sleep38168-21sed22:08:073
807026017,29sleep38168-21sed22:08:073
1184326017,22sleep20-21swapper/200:03:092
1184326017,22sleep20-21swapper/200:03:092
1003226015,28sleep210082-21/usr/sbin/munin23:58:002
1003226015,28sleep210082-21/usr/sbin/munin23:58:002
655325914,20sleep00-21swapper/022:03:080
655325914,20sleep00-21swapper/022:03:080
2979625915,27sleep229820-21/usr/sbin/munin23:18:082
2515425922,19sleep30-21swapper/323:03:083
2515425922,19sleep30-21swapper/323:03:083
2440925918,19sleep30-21swapper/323:02:393
2440925918,19sleep30-21swapper/323:02:393
1968425926,19sleep00-21swapper/021:03:060
1810525916,18sleep20-21swapper/219:17:402
1053725914,30sleep39797-21/usr/sbin/munin23:58:133
1053725914,30sleep39797-21/usr/sbin/munin23:58:133
697425815,26sleep36980-21ntp_states20:23:043
2806625815,26sleep028059-21apt-get19:47:540
2697425815,28sleep227010-21munin-run21:27:382
2536625820,20sleep20-21swapper/219:38:072
2160625814,20sleep00-21swapper/021:08:140
1888625818,21sleep30-21swapper/322:43:063
1888625818,21sleep30-21swapper/322:43:063
543225716,21sleep00-21swapper/020:18:050
521225714,26sleep25241-21irqstats20:17:592
1119325715,19sleep00-21swapper/020:37:390
3242925614,25sleep232423-21apt-get23:27:532
2833425618,17sleep30-21swapper/319:48:013
2761725615,21sleep30-21swapper/321:27:593
2133125617,18sleep30-21swapper/300:33:093
1508625615,23sleep015163-21threads00:13:080
1472525615,25sleep014720-21apt-get19:07:530
131392567,13sleep00-21swapper/019:02:530
1175225613,18sleep30-21swapper/300:03:063
1175225613,18sleep30-21swapper/300:03:063
676225513,24sleep30-21swapper/323:47:563
3164925512,16sleep30-21swapper/319:58:063
2783225517,19sleep00-21swapper/023:12:580
2773625515,18sleep00-21swapper/021:28:040
159825514,26sleep01734-21sed20:07:400
156862557,12sleep20-21swapper/222:33:042
1113725515,16sleep00-21swapper/000:02:380
1113725515,16sleep00-21swapper/000:02:380
1039125518,18sleep20-21swapper/222:17:382
725525419,17sleep30-21swapper/322:07:393
725525419,17sleep30-21swapper/322:07:393
313612546,11sleep20-21swapper/223:23:052
2598625413,24sleep226146-21cat23:07:392
1738125418,18sleep30-21swapper/300:22:393
1657125413,25sleep216647-21/usr/sbin/munin00:18:062
1373225414,24sleep313726-21apt-get22:27:523
399325315,19sleep20-21swapper/220:13:112
399325315,19sleep20-21swapper/220:13:112
347425317,22sleep33499-21fschecks_count20:12:593
347425317,22sleep33499-21fschecks_count20:12:593
2921225314,22sleep329254-21ntp_states21:33:043
2189325314,18sleep20-21swapper/222:53:022
2102225312,24sleep221018-21apt-get21:07:522
2006025313,17sleep20-21swapper/222:47:572
2005125313,23sleep020046-21apt-get19:22:510
189042534,37sleep30-21swapper/321:02:163
1582125313,23sleep215830-21munin-run00:17:372
1499225317,18sleep30-21swapper/300:13:073
1497425313,23sleep214997-21ntp_states20:48:062
1298425313,25sleep313021-21grep20:42:543
124672535,35sleep00-21swapper/020:39:450
539125216,21sleep00-21swapper/023:43:010
3095225213,22sleep330958-21df_inode23:22:543
2814925213,20sleep20-21swapper/223:13:042
2504525212,23sleep325054-21diskstats19:38:003
2462225214,22sleep00-21swapper/023:02:490
2462225214,22sleep00-21swapper/023:02:490
183925212,23sleep21835-21apt-get20:07:542
1419525219,18sleep00-21swapper/022:28:060
912325110,27sleep39129-21diskstats22:12:553
879425112,24sleep28822-21latency_hist22:12:382
879425112,24sleep28822-21latency_hist22:12:382
51012516,32sleep30-21swapper/323:42:473
485725116,20sleep30-21swapper/321:58:063
2860925112,22sleep228677-21awk19:48:082
2731625112,21sleep027414-21users19:43:090
1820425113,18sleep20-21swapper/200:23:072
2619425010,24sleep226190-21apt-get23:07:512
2619425010,24sleep226190-21apt-get23:07:512
2257325012,21sleep322568-21apt-get21:12:543
2108525011,23sleep021096-21/usr/sbin/munin00:33:010
1417325011,19sleep20-21swapper/222:28:042
513224912,22sleep25202-21df23:42:552
2011024912,21sleep320128-21df_inode19:22:543
1537324913,21sleep315379-21diskstats22:32:573
307424814,17sleep20-21swapper/221:53:012
230882487,22sleep20-21swapper/222:57:502
230882487,22sleep20-21swapper/222:57:502
182524815,16sleep00-21swapper/021:48:060
77324617,16sleep30-21swapper/320:03:063
2471324513,16sleep20-21swapper/223:02:552
163082448,16sleep20-21swapper/220:52:592
124512439,16sleep30-21swapper/322:23:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional