You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 07:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Sun Nov 23, 2025 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
156064123590,2sleep30-21swapper/321:26:213
1422216996644,19cyclictest1618229-21diskstats22:26:140
1422216996442,19cyclictest1707005-21latency_hist23:55:010
1422218996052,6cyclictest1543430-21kworker/u8:3+events_unbound21:36:291
15091092590,3sleep3586-21kworker/3:2+events20:36:173
1422218995954,3cyclictest1588156-21kworker/u8:0+flush-8:022:31:301
1422216995951,6cyclictest1637566-21latency_hist22:45:010
1422218995849,6cyclictest1746859-21latency_hist00:35:011
1422218995849,6cyclictest1687230-21latency_hist23:35:001
1422218995849,6cyclictest1687230-21latency_hist23:35:001
1422216995749,6cyclictest1711042-21ntpq23:56:250
1422216995749,5cyclictest1692163-21latency_hist23:40:000
1422218995651,3cyclictest1422075-21kworker/u8:0+flush-8:020:06:251
1422218995647,6cyclictest1543430-21kworker/u8:3+events_unbound22:01:241
1422218995349,3cyclictest583624-21kworker/u8:4+flush-8:022:11:281
1422218995243,7cyclictest1652931-21/usr/sbin/munin23:01:221
1422218995144,5cyclictest1588156-21kworker/u8:0+flush-8:023:11:261
1422216995130,19cyclictest1608063-21/usr/sbin/munin22:16:290
1422218995045,4cyclictest583624-21kworker/u8:4+flush-8:019:46:261
1422218994939,7cyclictest1427083-21latency_hist19:15:011
1422216994941,5cyclictest1572233-21ntpq21:36:250
1422218994843,3cyclictest1607987-21kworker/u8:1+events_unbound23:16:161
1422218994838,7cyclictest1602792-21latency_hist22:10:011
1422216994723,21cyclictest1641988-21ntpq22:46:260
1422216994721,24cyclictest1439751-21diskstats19:26:160
1422222994624,9cyclictest1702077-21cat23:50:012
1422218994637,7cyclictest583624-21kworker/u8:4+flush-8:023:51:251
1422216994628,16cyclictest1429202-21irqrtprio19:15:060
1422218994540,4cyclictest1365376-21kworker/u8:2+flush-8:019:16:151
1422218994534,8cyclictest1552282-21ntpq21:16:251
1422218994439,4cyclictest1543430-21kworker/u8:3+flush-8:021:26:351
1422216994416,8cyclictest1677374-21true23:25:000
1422222994325,3cyclictest1518618-21aten2_r5power_c20:46:132
1422218994338,4cyclictest1607987-21kworker/u8:1+events_unbound22:36:211
1422218994338,3cyclictest583624-21kworker/u8:4+events_unbound20:21:301
1422218994337,4cyclictest1697159-21kworker/u8:1+events_unbound00:16:301
1422218994335,6cyclictest1616800-21ntpq22:21:251
1422216994325,16cyclictest1657563-21cron23:05:000
1422216994320,21cyclictest282-21systemd-journal20:25:000
1422218994238,3cyclictest1588156-21kworker/u8:0+events_unbound00:26:291
1422218994236,4cyclictest1607987-21kworker/u8:1+events_unbound22:41:301
1422218994234,6cyclictest583624-21kworker/u8:4+flush-8:021:56:251
1422218994234,6cyclictest1578373-21/usr/sbin/munin21:46:241
1422218994232,8cyclictest1661640-21ntpq23:06:231
1422218994231,8cyclictest1735858-21ntpq00:21:261
1422216994233,7cyclictest1671527-21ntpq23:16:250
1422216994229,9cyclictest282-21systemd-journal21:53:420
1422216994222,17cyclictest1747415-21cpuspeed_turbos00:36:140
1422216994221,19cyclictest1482810-21ntpq20:06:270
1422216994218,6cyclictest1558078-21latency_hist21:25:010
1422218994136,4cyclictest1543430-21kworker/u8:3+flush-8:021:11:271
1422218994136,4cyclictest1365376-21kworker/u8:2+flush-8:019:51:141
1422216994134,5cyclictest1492600-21ntpq20:16:250
1422216994133,5cyclictest1592886-21latency_hist22:00:010
1422216994122,17cyclictest1543191-21cron21:10:000
1422222994023,2cyclictest1747222-21aten2_r5power_c00:36:132
1422218994034,4cyclictest1365376-21kworker/u8:2+flush-8:021:21:281
1422218994032,6cyclictest1677696-21apt-get23:26:121
1422218994031,7cyclictest1563554-21apt-get21:31:091
1422216994021,16cyclictest1717556-21fschecks_count00:06:130
1422216994020,18cyclictest1730885-21ntpq00:16:230
1422222993921,5cyclictest1707346-21apt-get23:56:132
1422222993919,7cyclictest282-21systemd-journal20:39:492
1422218993935,3cyclictest1588156-21kworker/u8:0+events_unbound23:46:301
1422218993934,4cyclictest1422075-21kworker/u8:0+flush-8:020:16:171
1422218993931,6cyclictest1449371-21apt-get19:36:111
1422216993922,15cyclictest1582685-21unixbench_singl21:46:320
1422218993833,3cyclictest583624-21kworker/u8:4+events_unbound20:26:161
1422218993831,5cyclictest1422075-21kworker/u8:0+events_unbound19:56:291
142221699386,10cyclictest0-21swapper/000:26:280
1422216993820,16cyclictest0-21swapper/021:30:010
1422216993819,17cyclictest1683532-21tr23:31:180
1422216993819,17cyclictest1683532-21tr23:31:180
1422222993729,6cyclictest1736828-21cp00:25:012
1422222993725,9cyclictest1592882-21awk22:00:002
1422218993733,3cyclictest1607987-21kworker/u8:1+flush-8:022:16:311
1422218993727,8cyclictest1438173-21ntpq19:21:281
1422216993729,5cyclictest1453971-21latency_hist19:40:010
1422216993718,17cyclictest1701756-21unixbench_singl23:46:320
1422216993717,3cyclictest405-21dbus-daemon20:50:000
1422216993715,7cyclictest1692492-21/usr/sbin/munin23:41:230
1422216993714,7cyclictest1716109-21ntpq00:01:270
1422218993631,4cyclictest583624-21kworker/u8:4+flush-8:022:56:131
1422218993631,4cyclictest583624-21kworker/u8:4+flush-8:020:31:311
1422218993631,4cyclictest1365376-21kworker/u8:2+flush-8:020:01:271
1422218993629,5cyclictest1607987-21kworker/u8:1+flush-8:022:26:311
1422218993629,5cyclictest1365376-21kworker/u8:2+events_unbound20:41:161
1422216993628,5cyclictest1473728-21latency_hist20:00:010
1422216993626,7cyclictest1607757-21latency_hist22:15:010
1422216993620,4cyclictest282-21systemd-journal00:14:080
1422216993620,2cyclictest1573698-21cpuspeed_turbos21:41:150
1422216993619,2cyclictest1438977-21systemd-run19:25:010
1422216993618,2cyclictest0-21swapper/020:11:320
1422216993617,4cyclictest1531746-21kthreadcore20:56:230
1422216993617,2cyclictest0-21swapper/023:06:230
1422216993617,2cyclictest0-21swapper/022:56:150
1422216993617,2cyclictest0-21swapper/020:55:000
1422218993529,4cyclictest1365376-21kworker/u8:2+events_unbound21:51:241
1422216993517,3cyclictest1-21systemd00:25:010
1422216993513,8cyclictest282-21systemd-journal19:32:200
1422216993415,4cyclictest1506172-21kthreadcore20:31:210
1422218993328,4cyclictest1588156-21kworker/u8:0+events_unbound23:21:131
1422218993326,5cyclictest583624-21kworker/u8:4+events_unbound23:41:251
1422218993325,6cyclictest1726843-21latency_hist00:15:011
1422216993323,8cyclictest1537421-21ntpq21:01:260
1422216993323,8cyclictest1537421-21ntpq21:01:250
1422216993318,2cyclictest1422211-21cyclictest22:09:500
1422216993317,4cyclictest1514262-21irqcore20:41:190
1422216993316,2cyclictest0-21swapper/021:20:010
1422216993314,4cyclictest0-21swapper/019:51:120
1422216993312,5cyclictest1513283-21latency_hist20:40:010
1422222993228,3cyclictest1687229-21cron23:35:002
1422222993228,3cyclictest1687229-21cron23:35:002
1422218993228,3cyclictest1365376-21kworker/u8:2+flush-8:020:36:131
1422218993227,3cyclictest1697159-21kworker/u8:1+events_unbound00:36:131
1422218993226,4cyclictest1422075-21kworker/u8:0+events_unbound20:56:261
1422218993224,5cyclictest1449033-21latency_hist19:35:011
1422216993223,7cyclictest1454328-21apt-get19:41:130
14219732324,9sleep20-21swapper/219:09:132
1422218993127,3cyclictest1607987-21kworker/u8:1+events_unbound22:51:331
1422218993127,3cyclictest1543430-21kworker/u8:3+events_unbound21:45:011
1422218993126,3cyclictest1458600-21unixbench_multi19:41:331
1422218993123,4cyclictest0-21swapper/123:36:101
1422216993023,5cyclictest1547798-21grep21:11:300
16369602290,2sleep3481ktimers/322:41:303
1422222992922,5cyclictest1652611-21awk23:00:002
142221699295,19cyclictest1385853-21kworker/0:2+events19:10:110
1422222992824,2cyclictest1681888-21unixbench_multi23:26:332
1422218992822,5cyclictest1588156-21kworker/u8:0+events_unbound00:01:211
1422218992821,5cyclictest583624-21kworker/u8:4+flush-8:021:06:351
1422216992822,4cyclictest1563844-21fschecks_count21:31:140
1422218992723,3cyclictest1422075-21kworker/u8:0+flush-8:020:11:161
1422218992722,4cyclictest1588156-21kworker/u8:0+flush-8:023:56:321
1422218992722,4cyclictest1422075-21kworker/u8:0+events_unbound20:46:301
1422218992718,7cyclictest1523498-21apt-get20:51:111
1422216992724,2cyclictest1623635-21irqrtprio22:31:190
1422216992724,2cyclictest1474299-21cpuspeed_turbos20:01:140
1422222992623,2cyclictest1633164-21fschecks_count22:41:152
1422218992622,3cyclictest1422075-21kworker/u8:0+events_unbound21:01:131
1422218992622,3cyclictest1422075-21kworker/u8:0+events_unbound21:01:131
1422216992622,3cyclictest1632539-21awk22:40:000
142222299252,2cyclictest16096782cut22:16:202
1422222992521,2cyclictest1623206-21fschecks_count22:31:162
1422222992521,2cyclictest1427085-21munin-plugin-st19:15:012
1422222992521,2cyclictest0-21swapper/221:55:002
1422222992520,3cyclictest1617260-21unixbench_multi22:21:312
1422222992518,5cyclictest1563082-21cp21:30:012
1422222992518,5cyclictest1518114-21journal-offline20:43:262
1422216992520,4cyclictest1463526-21unixbench_singl19:46:310
1422216992517,6cyclictest1617313-21grep22:21:320
14920132240,2sleep3481ktimers/320:16:233
14548062240,2sleep3586-21kworker/3:2+events19:41:193
1422222992421,2cyclictest1687594-21aten2_r5power_c23:36:112
1422222992421,2cyclictest1489196-21fschecks_time20:16:172
1422222992420,2cyclictest1721550-21unixbench_singl00:06:292
1422222992420,2cyclictest1628204-21fschecks_time22:36:172
1422222992419,3cyclictest1642385-21unixbench_multi22:46:302
1422222992416,6cyclictest1553223-21cat21:20:012
1422222992415,7cyclictest1726470-21unixbench_multi00:11:312
1422218992420,3cyclictest1588156-21kworker/u8:0+flush-8:000:05:361
1422218992419,3cyclictest1638358-21cpuspeed_turbos22:46:131
1422216992422,1cyclictest0-21swapper/023:11:290
15202392230,5sleep3481ktimers/320:46:213
142222299235,2cyclictest16662072grep23:11:212
1422222992320,2cyclictest1499088-21fschecks_time20:26:182
1422222992319,2cyclictest1429415-21aten2_r5power_p19:16:132
1422222992316,5cyclictest1657574-21turbostat.cron23:05:002
1422222992312,9cyclictest1528432-21munin-node20:56:112
1422218992318,3cyclictest1443788-21unixbench_singl19:26:321
14174682234,13sleep10-21swapper/119:06:111
142222299225,2cyclictest14517072cut19:36:282
142222299222,14cyclictest1598198-21apt-get22:06:112
1422222992219,2cyclictest1668094-21fschecks_time23:16:152
1422222992218,2cyclictest1464303-21aten2_r5power_v19:51:132
1422222992218,2cyclictest0-21swapper/221:40:012
1422222992217,3cyclictest1567852-21unixbench_singl21:31:292
1422222992217,3cyclictest1547800-21unixbench_multi21:11:302
1422216992218,2cyclictest0-21swapper/000:35:010
1422216992217,3cyclictest1644586-21kthreadcore22:51:220
1422216992215,4cyclictest0-21swapper/023:26:130
15739552210,3sleep3586-21kworker/3:2+events21:41:183
14502472210,2sleep3471rcuc/319:36:253
1422222992115,4cyclictest1741525-21unixbench_multi00:26:312
142222299208,8cyclictest1459281-21apt-get19:46:112
1422222992016,2cyclictest0-21swapper/220:31:292
1422222992015,3cyclictest1607394-21unixbench_multi22:11:302
1422222992015,3cyclictest1579300-21sed21:46:202
1422216992012,6cyclictest1502440-21grep20:26:250
142222299192,13cyclictest1523498-21apt-get20:51:132
1422222991916,2cyclictest1643383-21fschecks_count22:51:172
1422222991915,2cyclictest1658024-21aten2_r5power_v23:06:122
1422222991915,2cyclictest1454643-21fschecks_time19:41:182
1422222991915,2cyclictest0-21swapper/220:11:202
1422222991915,2cyclictest0-21swapper/219:31:262
1422222991914,3cyclictest1702522-21aten2_r5power_v23:51:132
1422222991910,7cyclictest1472431-21tr19:56:242
142222299190,5cyclictest1553582-21cut21:21:132
142221699192,13cyclictest1422054-21latency_hist19:10:010
1422216991915,2cyclictest0-21swapper/022:01:310
142222299188,7cyclictest1593242-21apt-get22:01:112
1422222991814,2cyclictest0-21swapper/200:31:312
14175242183,8sleep30-21swapper/319:06:133
142222299178,7cyclictest0-21swapper/223:21:282
142222299173,8cyclictest1542423-21ntpq21:06:272
142222299172,11cyclictest1692520-21apt-get23:41:112
1422222991713,2cyclictest0-21swapper/220:01:202
1422222991712,4cyclictest1711946-21kworker/u8:2+flush-8:000:05:012
142222299171,12cyclictest1439506-21taskset19:26:142
142222299171,11cyclictest1479699-21irqcore20:06:192
142222299168,6cyclictest1618199-21seq22:26:142
142222299168,6cyclictest1435407-21tr19:21:202
142222299162,13cyclictest0-21swapper/221:42:412
1422222991613,2cyclictest0-21swapper/200:15:322
1422222991612,2cyclictest0-21swapper/220:21:302
1422222991611,4cyclictest1534233-21sed21:01:182
1422222991611,4cyclictest1534233-21sed21:01:182
15882662152,8sleep30-21swapper/321:56:123
15291772153,8sleep30-21swapper/320:56:213
16930092143,8sleep30-21swapper/323:41:183
14440832142,8sleep30-21swapper/319:30:013
17216512132,7sleep30-21swapper/300:07:113
16624702132,7sleep30-21swapper/323:10:003
16534282133,7sleep30-21swapper/323:01:173
16525832132,6sleep30-21swapper/323:00:013
16399572132,7sleep30-21swapper/322:46:203
16167582132,7sleep30-21swapper/322:21:243
15936982131,7sleep30-21swapper/322:01:173
15432272132,6sleep30-21swapper/321:10:003
14796962133,7sleep30-21swapper/320:06:213
17510292122,7sleep30-21swapper/300:36:283
17466242122,6sleep30-21swapper/300:32:153
17376482122,6sleep30-21swapper/300:26:173
17321232122,6sleep30-21swapper/300:21:153
17309302123,6sleep30-21swapper/300:16:263
17077202122,6sleep30-21swapper/323:56:193
16863182122,6sleep30-21swapper/323:31:233
16863182122,6sleep30-21swapper/323:31:233
16815092122,6sleep30-21swapper/323:26:293
16314932122,6sleep30-21swapper/322:36:253
16265062122,6sleep30-21swapper/322:31:243
16219462122,6sleep30-21swapper/322:26:303
16055862122,6sleep30-21swapper/322:11:213
15982012122,6sleep30-21swapper/322:06:113
15878102122,6sleep30-21swapper/321:53:433
15792822122,6sleep30-21swapper/321:46:213
15534812122,6sleep30-21swapper/321:21:133
15240162122,6sleep30-21swapper/320:51:193
15040802122,6sleep30-21swapper/320:31:173
15030732122,6sleep30-21swapper/320:26:353
14937812122,6sleep30-21swapper/320:20:493
14778352122,6sleep30-21swapper/320:01:273
14691102122,6sleep30-21swapper/319:56:133
14687732122,6sleep30-21swapper/319:55:013
14627822122,6sleep30-21swapper/319:46:223
14388192122,6sleep30-21swapper/319:23:473
14296992122,6sleep30-21swapper/319:16:163
14228432122,6sleep30-21swapper/319:11:193
17267492112,6sleep30-21swapper/300:14:013
17127162112,5sleep30-21swapper/300:01:213
17023802112,5sleep30-21swapper/323:51:093
17020132112,6sleep30-21swapper/323:49:253
16875962112,5sleep30-21swapper/323:36:113
16727182112,5sleep30-21swapper/323:21:123
16682532112,5sleep30-21swapper/323:16:183
16631332112,5sleep30-21swapper/323:11:163
16469312112,5sleep30-21swapper/322:51:293
16080622112,5sleep30-21swapper/322:16:113
15684042112,5sleep30-21swapper/321:35:053
15640072112,5sleep30-21swapper/321:31:163
15484522112,6sleep30-21swapper/321:15:393
15428042112,5sleep30-21swapper/321:06:323
15341272112,5sleep30-21swapper/321:01:183
15341272112,5sleep30-21swapper/321:01:183
15135192113,5sleep30-21swapper/320:40:033
14839512112,5sleep30-21swapper/320:11:143
14443542112,6sleep30-21swapper/319:30:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional