You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 20:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Tue Apr 28, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
161464523450,3sleep3451irq_work/311:01:083
1383480996755,4cyclictest1664206-21kworker/u8:1+events_unbound11:51:210
1383486996457,5cyclictest1479764-21kworker/u8:0+flush-8:009:16:201
1383480996155,4cyclictest4143982-21kworker/u8:3+events_unbound08:36:180
1383480996053,5cyclictest4143982-21kworker/u8:3+events_unbound10:01:180
1383480995953,4cyclictest4143982-21kworker/u8:3+events_unbound09:21:230
1383480995953,4cyclictest1383199-21kworker/u8:0+events_unbound07:15:020
1383480995751,4cyclictest1618923-21kworker/u8:3+events_unbound12:01:190
1383480995652,3cyclictest1618923-21kworker/u8:3+events_unbound11:41:230
1383480995650,4cyclictest1430186-21kworker/u8:1+events_unbound08:41:200
1383480995449,4cyclictest1553612-21kworker/u8:2+events_unbound11:10:020
1383480995449,3cyclictest1420210-21kworker/u8:2+events_unbound08:11:230
1383480995448,4cyclictest4143982-21kworker/u8:3+events_unbound07:41:230
1383480995448,4cyclictest1553612-21kworker/u8:2+events_unbound10:16:180
1383480995349,3cyclictest1430186-21kworker/u8:1+flush-8:008:51:150
14942272520,2chrt1494229-21grep08:56:250
1383480995248,3cyclictest1618923-21kworker/u8:3+events_unbound11:36:200
1383480995247,4cyclictest1383199-21kworker/u8:0+events_unbound07:11:230
1383480995247,3cyclictest1479764-21kworker/u8:0+events_unbound09:01:090
1383486995140,8cyclictest1573929-21latency_hist10:20:011
1383480995147,3cyclictest1664206-21kworker/u8:1+events_unbound11:56:200
1383480995147,3cyclictest1573640-21kworker/u8:0+flush-8:011:01:210
1383480995147,3cyclictest1518495-21kworker/u8:2+events_unbound09:31:200
1383480995046,3cyclictest1553612-21kworker/u8:2+flush-8:011:16:200
1383480995045,3cyclictest1420210-21kworker/u8:2+events_unbound08:21:090
1383480995044,4cyclictest4143982-21kworker/u8:3+events_unbound07:51:150
1383480994945,3cyclictest1479764-21kworker/u8:0+events_unbound09:11:190
1383480994944,4cyclictest1430186-21kworker/u8:1+flush-8:009:51:200
1383491994841,5cyclictest1599121-21cp10:45:012
1383480994844,3cyclictest4143982-21kworker/u8:3+events_unbound09:46:220
1383480994843,4cyclictest1618923-21kworker/u8:3+events_unbound11:26:220
1383480994842,5cyclictest1518495-21kworker/u8:2+events_unbound09:30:000
1383480994840,6cyclictest1648761-21kworker/u8:0+events_unbound12:10:000
1383480994840,3cyclictest1479764-21kworker/u8:0+events_unbound08:46:240
1383480994837,9cyclictest280-21systemd-journal07:20:380
1383491994710,21cyclictest1455372-21apt-get08:21:042
1383486994738,7cyclictest1387566-21ntpq07:11:171
1383486994737,7cyclictest1618026-21ntpq11:01:151
1383480994743,3cyclictest4143982-21kworker/u8:3+events_unbound09:56:190
1383480994743,3cyclictest1553612-21kworker/u8:2+flush-8:010:06:180
1383480994743,3cyclictest1553612-21kworker/u8:2+flush-8:010:06:180
1383480994742,4cyclictest1553612-21kworker/u8:2+events_unbound10:21:200
1383480994742,3cyclictest1618923-21kworker/u8:3+events_unbound11:46:170
1383480994741,4cyclictest4143982-21kworker/u8:3+events_unbound08:35:010
1383480994739,3cyclictest1664206-21kworker/u8:1+events_unbound12:22:390
1383491994639,5cyclictest1623958-21cp11:10:012
1383480994640,5cyclictest4143982-21kworker/u8:3+events_unbound07:36:170
1383480994640,4cyclictest1693785-21kworker/u8:2+events_unbound12:26:090
1383480994640,4cyclictest1430186-21kworker/u8:1+events_unbound09:06:060
1383480994541,3cyclictest1618923-21kworker/u8:3+events_unbound12:36:230
1383480994541,3cyclictest1479764-21kworker/u8:0+flush-8:009:16:220
1383480994541,3cyclictest1420210-21kworker/u8:2+flush-8:008:01:220
1383480994541,3cyclictest1420210-21kworker/u8:2+flush-8:007:56:200
1383480994537,5cyclictest280-21systemd-journal07:30:000
1383480994440,3cyclictest1579140-21kworker/u8:1+flush-8:010:26:190
1383480994440,3cyclictest1430186-21kworker/u8:1+events_unbound08:06:240
1383480994440,3cyclictest1430186-21kworker/u8:1+events_unbound08:06:240
1383480994439,3cyclictest1430186-21kworker/u8:1+events_unbound09:40:370
1383480994438,4cyclictest1573640-21kworker/u8:0+events_unbound10:41:060
1383486994339,3cyclictest1420210-21kworker/u8:2+flush-8:008:26:121
1383486994337,4cyclictest1579140-21kworker/u8:1+flush-8:010:56:231
1383480994339,3cyclictest1618923-21kworker/u8:3+flush-8:011:06:100
1383480994339,3cyclictest1430186-21kworker/u8:1+events_unbound08:16:210
1383486994238,3cyclictest1479764-21kworker/u8:0+flush-8:008:46:061
1383480994238,3cyclictest1579140-21kworker/u8:1+events_unbound10:51:270
1383480994237,4cyclictest1634026-21kworker/u8:1+events_unbound11:21:210
1383480994234,3cyclictest1383199-21kworker/u8:0+events_unbound07:46:390
1383486994131,7cyclictest280-21systemd-journal10:10:001
1383486994131,7cyclictest280-21systemd-journal10:10:001
1383480994137,3cyclictest1618923-21kworker/u8:3+events_unbound12:11:180
1383480994137,3cyclictest1618923-21kworker/u8:3+events_unbound12:11:180
1383480994137,3cyclictest1395748-21kworker/u8:2+events_unbound07:31:110
15718412400,3sleep375-21kworker/3:1+events10:16:133
1383486994031,7cyclictest1469019-21ntpq08:31:171
1383480994031,4cyclictest1553612-21kworker/u8:2+events_unbound10:31:100
15004512390,7sleep21500455-21grep09:06:102
1383486993933,4cyclictest1420210-21kworker/u8:2+flush-8:007:56:201
1383486993931,6cyclictest280-21systemd-journal08:02:381
1383486993930,7cyclictest1394484-21ntpq07:16:201
1383480993931,3cyclictest1618923-21kworker/u8:3+events_unbound12:18:400
1383480993931,3cyclictest1618923-21kworker/u8:3+events_unbound12:18:400
1383486993832,5cyclictest4143982-21kworker/u8:3+flush-8:009:21:181
1383486993832,4cyclictest1504807-21apt-get09:11:031
1383486993831,5cyclictest1449238-21ntpq08:11:181
1383480993833,4cyclictest1579140-21kworker/u8:1+events_unbound10:55:400
1383480993830,3cyclictest1618923-21kworker/u8:3+events_unbound12:31:240
1383491993729,6cyclictest280-21systemd-journal07:51:382
1383491993710,7cyclictest1678006-21ntpq12:01:162
1383486993730,5cyclictest1594055-21apt-get10:41:031
1383486993729,6cyclictest1664268-21apt-get11:51:021
1383480993733,3cyclictest1553612-21kworker/u8:2+events_unbound10:11:240
1383480993733,3cyclictest1518495-21kworker/u8:2+events_unbound09:35:070
1383480993729,3cyclictest1573640-21kworker/u8:0+events_unbound10:37:150
1383491993611,6cyclictest1689118-21apt-get12:16:032
1383491993611,6cyclictest1689118-21apt-get12:16:032
1383486993632,3cyclictest4143982-21kworker/u8:3+events_unbound07:51:231
1383486993531,3cyclictest1430186-21kworker/u8:1+events_unbound09:56:091
1383486993529,4cyclictest1553612-21kworker/u8:2+events_unbound11:41:191
1383486993528,5cyclictest280-21systemd-journal08:43:381
1383480993528,2cyclictest1618923-21kworker/u8:3+events_unbound11:31:390
1383480993527,3cyclictest1579140-21kworker/u8:1+events_unbound10:46:450
1383491993318,8cyclictest1554464-21apt-get10:01:032
1383491993318,10cyclictest1669247-21apt-get11:56:042
1383491993218,9cyclictest1519629-21apt-get09:26:042
1383486993226,5cyclictest4143982-21kworker/u8:3+flush-8:010:11:161
1383486993226,5cyclictest1553612-21kworker/u8:2+events_unbound11:36:151
1383486993226,4cyclictest1618923-21kworker/u8:3+flush-8:012:06:231
138349199315,13cyclictest1420517-21apt-get07:46:042
1383491993125,4cyclictest1713544-21turbostat.cron12:40:012
1383491993125,4cyclictest1668574-21unixbench_singl11:51:222
1383491993121,4cyclictest1390660-21apt-get07:16:072
1383486993128,2cyclictest1618923-21kworker/u8:3+flush-8:012:01:131
1383486993127,3cyclictest1693754-21systemctl12:20:001
1383486993127,3cyclictest1693754-21systemctl12:19:591
1383486993127,2cyclictest1479764-21kworker/u8:0+events_unbound09:00:011
1383486993126,4cyclictest1430186-21kworker/u8:1+events_unbound09:51:071
1383486993125,4cyclictest4143982-21kworker/u8:3+events_unbound10:01:191
1383486993123,6cyclictest1608786-21apt-get10:55:001
138348699310,29cyclictest1-21systemd07:20:371
1383480993128,2cyclictest1420210-21kworker/u8:2+flush-8:008:29:530
138349199308,6cyclictest1419578-21/usr/sbin/munin07:41:212
1383491993024,4cyclictest1499533-21turbostat.cron09:05:002
1383491993021,7cyclictest1440550-21apt-get08:06:032
1383491993021,7cyclictest1440550-21apt-get08:06:032
1383491993020,3cyclictest1539413-21apt-get09:46:052
1383491993019,5cyclictest1644252-21apt-get11:31:032
1383491993017,8cyclictest1435615-21apt-get08:01:042
1383491993015,9cyclictest1460286-21apt-get08:26:032
1383491993013,10cyclictest1654169-21apt-get11:41:062
1383491993010,6cyclictest1400686-21apt-get07:26:052
1383491993010,12cyclictest0-21swapper/209:41:182
1383486993026,3cyclictest1593695-21cron10:40:001
1383486993026,2cyclictest1644971-21irqrtprio11:31:091
1383486993025,3cyclictest1618923-21kworker/u8:3+events_unbound12:36:111
1383486993024,4cyclictest1624425-21apt-get11:11:031
1383486993022,6cyclictest1694093-21apt-get12:21:001
1383486993021,7cyclictest1618923-21kworker/u8:3+flush-8:012:11:171
1383486993021,7cyclictest1618923-21kworker/u8:3+flush-8:012:11:171
138349199298,4cyclictest1687981-21ntpq12:11:182
138349199298,4cyclictest1687981-21ntpq12:11:182
138349199297,14cyclictest0-21swapper/211:26:212
138349199297,12cyclictest1573145-21ntpq10:16:192
1383491992923,4cyclictest1604215-21aten2_r5power_e10:51:062
1383491992919,8cyclictest280-21systemd-journal11:05:012
1383486992925,3cyclictest1430186-21kworker/u8:1+events_unbound08:16:151
1383486992924,4cyclictest1579140-21kworker/u8:1+events_unbound10:26:201
1383486992922,5cyclictest280-21systemd-journal11:30:001
1383486992920,6cyclictest1420192-21latency_hist07:45:011
138349199287,11cyclictest1563220-21ntpq10:06:192
138349199287,11cyclictest1563220-21ntpq10:06:192
1383491992824,2cyclictest1574329-21aten2_r5power_e10:21:042
1383491992824,2cyclictest1405913-21fschecks_time07:31:082
1383491992818,4cyclictest1509738-21apt-get09:16:042
1383486992823,4cyclictest1383199-21kworker/u8:0+events_unbound07:46:051
1383486992822,4cyclictest1663701-21apt11:50:001
1383486992821,5cyclictest1499866-21apt-get09:06:031
1383486992821,5cyclictest1420210-21kworker/u8:2+flush-8:008:21:201
138349199276,3cyclictest1652946-21ntpq11:36:142
138349199276,11cyclictest1697928-21ntpq12:21:152
138349199275,5cyclictest1544343-21apt-get09:51:032
1383491992721,4cyclictest1708176-21unixbench_multi12:31:222
1383491992721,4cyclictest1579262-21aten2_r5power_e10:26:032
1383491992719,3cyclictest1480062-21apt-get08:46:032
1383491992718,4cyclictest1599242-21apt-get10:46:042
1383491992718,4cyclictest1383784-21apt-get07:11:052
1383486992723,2cyclictest1539104-21systemd-run09:45:001
1383486992720,5cyclictest280-21systemd-journal09:35:381
138349199265,11cyclictest1445482-21apt-get08:11:032
1383491992620,4cyclictest1634373-21aten2_r5power_c11:21:052
1383491992615,9cyclictest1682920-21/usr/sbin/munin12:06:172
1383491992615,8cyclictest1478891-21ntpq08:41:162
1383491992614,8cyclictest1489979-21apt-get08:56:062
1383491992612,11cyclictest1568829-21runrttasks10:13:292
1383491992611,10cyclictest1609347-21apt-get10:56:042
1383486992622,3cyclictest4143982-21kworker/u8:3+events_unbound08:36:151
1383486992622,3cyclictest1553612-21kworker/u8:2+flush-8:010:31:221
1383486992622,2cyclictest1410723-21cpuspeed07:36:071
1383486992621,4cyclictest1579140-21kworker/u8:1+events_unbound10:50:011
1383486992621,4cyclictest1553612-21kworker/u8:2+flush-8:011:16:171
1383486992621,4cyclictest1553612-21kworker/u8:2+events_unbound10:21:141
1383486992620,5cyclictest1618923-21kworker/u8:3+events_unbound11:06:161
1383491992521,2cyclictest1505025-21cpuspeed_turbos09:11:072
1383491992517,4cyclictest1470198-21apt-get08:36:042
1383491992513,8cyclictest1488779-21ntpq08:51:162
1383486992521,3cyclictest1618923-21kworker/u8:3+flush-8:011:21:141
1383486992519,4cyclictest1489654-21latency_hist08:55:011
1383486992515,6cyclictest0-21swapper/107:31:041
16007952240,8sleep30-21swapper/310:46:113
138349199245,5cyclictest1398441-21cut07:21:112
1383491992420,2cyclictest1450682-21fschecks_count08:16:072
1383491992420,2cyclictest0-21swapper/211:50:002
1383491992411,11cyclictest1519024-21chrt09:21:352
1383491992411,10cyclictest1589048-21chrt10:35:432
1383486992420,3cyclictest1420210-21kworker/u8:2+events_unbound08:06:201
1383486992420,3cyclictest1420210-21kworker/u8:2+events_unbound08:06:191
1383486992420,2cyclictest0-21swapper/107:10:001
1383486992418,4cyclictest0-21swapper/112:00:001
1383486992417,6cyclictest1430186-21kworker/u8:1+flush-8:009:46:181
138349199239,12cyclictest1628828-21chrt11:11:582
138349199235,8cyclictest0-21swapper/207:57:122
138349199235,4cyclictest0-21swapper/210:32:332
138349199235,4cyclictest0-21swapper/209:36:072
138349199234,4cyclictest0-21swapper/207:36:062
1383491992319,2cyclictest1629625-21cpuspeed_turbos11:16:052
1383491992317,4cyclictest0-21swapper/207:10:012
1383486992319,3cyclictest1693785-21kworker/u8:2+events_unbound12:31:051
1383486992318,4cyclictest1479764-21kworker/u8:0+flush-8:009:01:091
1383486992312,3cyclictest0-21swapper/112:26:231
13785592232,3sleep00-21swapper/007:05:420
138349199225,4cyclictest0-21swapper/209:55:012
138349199225,4cyclictest0-21swapper/208:31:152
138349199224,3cyclictest0-21swapper/209:31:062
1383491992210,9cyclictest1699057-21seq12:26:042
138348699228,5cyclictest0-21swapper/109:28:141
1383486992218,2cyclictest0-21swapper/109:31:061
1383486992217,3cyclictest0-21swapper/107:26:131
13829312184,8sleep30-21swapper/307:07:183
14155442172,9sleep30-21swapper/307:41:053
15837122146,5sleep30-21swapper/310:28:383
15157032140,3chrt75-21kworker/3:1-events09:21:123
14591262140,2sleep3471rcuc/308:21:173
16940972132,7sleep30-21swapper/312:21:023
16189292132,6sleep30-21swapper/311:05:013
15430062136,5sleep30-21swapper/309:46:173
15134582132,7sleep30-21swapper/309:16:143
14886352132,7sleep30-21swapper/308:51:153
14698982132,6sleep30-21swapper/308:35:023
17084242122,6sleep30-21swapper/312:33:263
16842972125,4sleep30-21swapper/312:11:073
16842972125,4sleep30-21swapper/312:11:063
16744752122,6sleep30-21swapper/312:01:083
16642052122,6sleep30-21swapper/311:50:363
16504342123,6sleep30-21swapper/311:36:123
16440142122,5sleep30-21swapper/311:30:023
16402062123,6sleep30-21swapper/311:26:133
16389742121,7sleep30-21swapper/311:25:003
16298752122,6sleep30-21swapper/311:16:093
16290602122,5sleep30-21swapper/311:15:013
16083902122,6sleep30-21swapper/310:51:253
15937472122,6sleep30-21swapper/310:40:023
15894682122,6sleep30-21swapper/310:36:093
15746422122,6sleep30-21swapper/310:21:083
15643322124,5sleep30-21swapper/310:11:053
15492802122,7sleep30-21swapper/309:56:043
15483532122,6sleep30-21swapper/309:51:223
15292092122,5sleep30-21swapper/309:35:013
15044562122,5sleep30-21swapper/309:10:013
14946672122,5sleep30-21swapper/309:00:013
14753802122,6sleep30-21swapper/308:41:083
14608092122,6sleep30-21swapper/308:26:113
14507632122,6sleep30-21swapper/308:16:083
14409222122,6sleep30-21swapper/308:06:093
14409222122,6sleep30-21swapper/308:06:093
14308662122,6sleep30-21swapper/307:56:103
14149792122,6sleep30-21swapper/307:37:403
14052852122,5sleep30-21swapper/307:30:013
14009342122,6sleep30-21swapper/307:26:073
13955432122,6sleep30-21swapper/307:20:023
17089512112,5sleep30-21swapper/312:36:073
17002702111,6sleep30-21swapper/312:26:123
16788912112,5sleep30-21swapper/312:05:013
16691192111,6sleep30-21swapper/311:55:023
16590422112,5sleep30-21swapper/311:45:393
16538022112,5sleep30-21swapper/311:40:013
16092492112,5sleep30-21swapper/310:55:113
15844112112,5sleep30-21swapper/310:31:073
15593972112,5sleep30-21swapper/310:06:033
15593972112,5sleep30-21swapper/310:06:023
15544382112,5sleep30-21swapper/310:01:013
15351662112,5sleep30-21swapper/309:41:113
15289592113,5sleep30-21swapper/309:32:083
15236462112,6sleep30-21swapper/309:26:223
15007622112,5sleep30-21swapper/309:06:123
14805852112,5sleep30-21swapper/308:46:103
14655682112,5sleep30-21swapper/308:31:093
14454492112,6sleep30-21swapper/308:10:573
14365002112,5sleep30-21swapper/308:01:103
14254002112,5sleep30-21swapper/307:50:283
14204142112,5sleep30-21swapper/307:45:023
13911372112,5sleep30-21swapper/307:16:133
13838702112,5sleep30-21swapper/307:11:063
16890152102,5sleep30-21swapper/312:15:043
16890152102,5sleep30-21swapper/312:15:043
14898812102,5sleep30-21swapper/308:55:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional