You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-31 - 05:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sun May 31, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
134369622080,3sleep375-21kworker/3:1+mm_percpu_wq00:30:583
11749132850,6chrt0-21swapper/221:40:552
1020009997160,8cyclictest1151220-21/usr/sbin/munin21:20:591
13314162560,3sleep30-21swapper/300:20:463
1020004995646,8cyclictest1244219-21apt-get22:50:520
1020009995345,6cyclictest1116619-21/usr/sbin/munin20:45:551
1020009995345,6cyclictest1116619-21/usr/sbin/munin20:45:541
1020009995344,7cyclictest1024883-21latency_hist19:15:001
1020010995145,5cyclictest1279700-21kworker/u8:2+flush-8:000:20:512
1020009995141,8cyclictest1268900-21ntpq23:15:481
1020010995045,4cyclictest1225185-21kworker/u8:1+flush-8:022:50:562
1020009994944,4cyclictest1140687-21kworker/u8:2+flush-8:021:15:541
1020009994938,9cyclictest1704890-21systemd-journal19:45:011
1020010994842,4cyclictest666912-21kworker/u8:4+flush-8:020:10:582
1020009994839,7cyclictest1161087-21/usr/sbin/munin21:30:561
1020009994837,9cyclictest1036084-21ntpq19:20:491
1020010994742,4cyclictest666912-21kworker/u8:4+flush-8:020:15:502
1020009994742,3cyclictest1230227-21kworker/u8:3+flush-8:023:25:561
1020004994738,7cyclictest1086669-21apt-get20:15:350
1020009994638,6cyclictest1136390-21apt-get21:05:381
1020009994539,5cyclictest1076200-21kworker/u8:0+events_unbound20:55:551
1020009994536,7cyclictest1042198-21/usr/sbin/munin19:30:491
1020009994534,9cyclictest1060795-21ntpq19:45:501
1020004994535,8cyclictest1105522-21ntpq20:30:510
1020009994437,5cyclictest662606-21kworker/u8:1+flush-8:019:25:551
1020004994437,5cyclictest1076200-21kworker/u8:0+flush-8:021:05:420
1020004994435,7cyclictest1704890-21systemd-journal19:25:000
1020004994435,7cyclictest1348270-21ntpq00:35:510
1020004994434,8cyclictest1100327-21ntpq20:25:510
1020010994338,4cyclictest1334111-21kworker/u8:1+flush-8:000:25:512
1020009994335,6cyclictest1329416-21/usr/sbin/munin00:20:491
1020004994234,6cyclictest1704890-21systemd-journal21:40:010
1020004994231,8cyclictest1339278-21apt-key00:30:010
1020010994133,6cyclictest1704890-21systemd-journal00:06:572
1020009994137,3cyclictest1076200-21kworker/u8:0+flush-8:020:50:431
1020009994136,4cyclictest1225185-21kworker/u8:1+events_unbound23:55:571
1020009994134,6cyclictest1225185-21kworker/u8:1+flush-8:022:45:521
1020009994133,6cyclictest1313461-21ntpq00:00:541
1020004994136,4cyclictest1076200-21kworker/u8:0+events_unbound21:35:000
1020009994035,3cyclictest1230227-21kworker/u8:3+events_unbound22:55:551
1020009994034,5cyclictest666912-21kworker/u8:4+flush-8:021:40:521
1020009994033,5cyclictest666912-21kworker/u8:4+flush-8:021:50:511
1020009994031,6cyclictest1220297-21latency_hist22:30:001
1020004994033,5cyclictest1031790-21apt-get19:20:000
1020010993935,3cyclictest1019856-21kworker/u8:2+events_unbound19:20:482
1020009993935,3cyclictest1076200-21kworker/u8:0+flush-8:021:00:541
1020009993930,7cyclictest1050864-21ntpq19:35:511
1020009993834,3cyclictest1076200-21kworker/u8:0+flush-8:020:25:421
1020009993833,3cyclictest1230227-21kworker/u8:3+flush-8:023:05:511
1020009993830,6cyclictest1704890-21systemd-journal20:34:591
1020004993833,4cyclictest666912-21kworker/u8:4+flush-8:019:35:550
1020004993833,4cyclictest1140687-21kworker/u8:2+flush-8:022:25:560
1020009993731,4cyclictest1328271-21ntpq00:15:501
1020009993731,4cyclictest1260241-21apt-get23:10:371
1020009993729,6cyclictest1704890-21systemd-journal23:47:571
1020009993729,5cyclictest1066603-21latency_hist19:55:011
1020004993729,6cyclictest1259948-21latency_hist23:10:010
1020004993728,7cyclictest1111664-21/usr/sbin/munin20:40:530
1020009993630,5cyclictest1225185-21kworker/u8:1+flush-8:023:50:531
1020009993628,6cyclictest1111689-21apt-get20:40:381
1020004993631,3cyclictest1140687-21kworker/u8:2+flush-8:021:15:570
10152182364,10sleep10-21swapper/119:05:121
10152182364,10sleep10-21swapper/119:05:121
1020010993531,3cyclictest1076200-21kworker/u8:0+flush-8:020:50:462
1020010993524,8cyclictest1250057-21awk23:00:002
1020009993529,5cyclictest1076200-21kworker/u8:0+events_unbound20:05:401
1020009993527,6cyclictest1283780-21ntpq23:30:511
1020009993427,5cyclictest1140687-21kworker/u8:2+flush-8:021:45:561
1020009993329,3cyclictest1309622-21kworker/u8:3+events_unbound00:25:591
1020009993329,3cyclictest1106356-21kworker/u8:1+events_unbound20:35:501
1020009993327,4cyclictest1339541-21apt-get00:30:361
1020009993227,4cyclictest662606-21kworker/u8:1+events_unbound19:15:451
1020009993227,4cyclictest1230227-21kworker/u8:3+events_unbound23:20:411
1020009993226,5cyclictest1076200-21kworker/u8:0+events_unbound22:35:541
1020009993225,5cyclictest1704890-21systemd-journal20:01:561
1020009993223,7cyclictest1210197-21latency_hist22:20:001
1020009993221,9cyclictest1095400-21ntpq20:20:511
1020004993228,3cyclictest1140687-21kworker/u8:2+events_unbound22:15:400
1020004993227,3cyclictest1020191-21apt-get19:10:370
1020004993226,4cyclictest1200884-21fschecks_count22:10:390
1020004993226,4cyclictest1200884-21fschecks_count22:10:390
1020010993127,3cyclictest1304393-21kworker/u8:0+flush-8:000:00:482
1020010993123,6cyclictest1704890-21systemd-journal20:49:582
1020010993123,6cyclictest1704890-21systemd-journal20:49:572
1020009993125,4cyclictest1309622-21kworker/u8:3+flush-8:000:35:521
1020004993127,3cyclictest666912-21kworker/u8:4+events_unbound19:40:570
1020004993125,4cyclictest1230227-21kworker/u8:3+flush-8:023:15:550
1020009993026,3cyclictest1225185-21kworker/u8:1+flush-8:022:50:531
1020009993026,3cyclictest1187960-21kthreadcore21:55:461
1020009993026,2cyclictest1168192-21kthreadcore21:35:461
1020009993025,3cyclictest1290203-21fschecks_time23:40:421
1020009993025,3cyclictest1290203-21fschecks_time23:40:411
1020009993022,6cyclictest1309622-21kworker/u8:3+flush-8:000:05:511
1020004993024,4cyclictest1324209-21systemd-run00:15:000
1020010992923,4cyclictest662606-21kworker/u8:1+flush-8:019:19:592
1020010992922,5cyclictest1101517-21gpgv20:30:012
1020010992921,6cyclictest1704890-21systemd-journal23:25:572
1020010992918,8cyclictest1116358-21awk20:45:002
1020009992925,3cyclictest666912-21kworker/u8:4+flush-8:021:25:571
1020009992925,3cyclictest1225185-21kworker/u8:1+events_unbound23:35:381
1020009992925,3cyclictest1019998-21cyclictest23:00:431
1020009992923,5cyclictest1140687-21kworker/u8:2+flush-8:022:05:521
1020009992923,4cyclictest1225185-21kworker/u8:1+events_unbound22:40:411
1020009992923,4cyclictest1076200-21kworker/u8:0+events_unbound22:20:571
1020009992923,4cyclictest1076200-21kworker/u8:0+events_unbound22:20:571
1020009992922,5cyclictest1085488-21ntpq20:10:511
1020004992922,5cyclictest1071562-21latency_hist20:00:000
1020004992920,7cyclictest1704890-21systemd-journal19:50:000
102000499291,4cyclictest1249718-21users22:55:550
1020010992822,4cyclictest666912-21kworker/u8:4+events_unbound21:15:052
1020009992822,4cyclictest666912-21kworker/u8:4+events_unbound19:55:411
1020009992818,8cyclictest1323304-21ntpq00:10:511
1020004992826,1cyclictest1279700-21kworker/u8:2+events_unbound00:15:550
1020004992824,3cyclictest1076200-21kworker/u8:0+flush-8:020:20:570
1020004992824,2cyclictest0-21swapper/023:40:000
1020004992823,4cyclictest1309622-21kworker/u8:3+flush-8:000:06:290
1020004992822,4cyclictest1140687-21kworker/u8:2+flush-8:021:45:520
1020004992822,4cyclictest1086024-21unixbench_singl20:10:580
1020004992821,6cyclictest1225185-21kworker/u8:1+events_unbound23:45:530
1020004992818,7cyclictest1225234-21latency_hist22:35:020
10196672283,7sleep20-21swapper/219:08:032
10196672283,7sleep20-21swapper/219:08:032
10453082270,4sleep00-21swapper/019:30:470
1020009992722,4cyclictest1076200-21kworker/u8:0+flush-8:022:10:531
1020009992722,4cyclictest1076200-21kworker/u8:0+flush-8:022:10:531
1020009992721,4cyclictest1140687-21kworker/u8:2+events_unbound22:30:551
1020009992720,5cyclictest666912-21kworker/u8:4+flush-8:021:10:521
1020004992723,2cyclictest0-21swapper/022:25:000
1020004992723,2cyclictest0-21swapper/022:25:000
1020004992722,4cyclictest1225185-21kworker/u8:1+events_unbound23:50:560
1020004992722,3cyclictest1077020-21cpuspeed_turbos20:05:400
1020004992719,6cyclictest1185886-21aten2_r5power_p21:55:390
1020004992719,6cyclictest1125857-21unixbench_singl20:50:580
1020010992621,3cyclictest1204725-21timerandwakeup22:10:542
1020010992621,3cyclictest1204725-21timerandwakeup22:10:542
1020010992620,4cyclictest1199907-21unixbench_multi22:05:572
1020009992622,3cyclictest1140687-21kworker/u8:2+events_unbound22:00:371
1020004992622,2cyclictest0-21swapper/020:00:570
1020004992621,3cyclictest1235185-21sh22:45:000
1020004992621,3cyclictest0-21swapper/019:50:450
1020004992620,4cyclictest1140687-21kworker/u8:2+events_unbound22:05:560
1020004992619,6cyclictest1274420-21unixbench_singl23:20:580
1020004992619,6cyclictest1106356-21kworker/u8:1+flush-8:021:01:290
1020004992521,3cyclictest666912-21kworker/u8:4+events_unbound21:20:510
1020004992519,4cyclictest1313970-21unixbench_singl00:01:000
1020004992519,4cyclictest1159107-21kthreadcore21:25:490
1020004992518,5cyclictest1260326-21aten2_r5power_p23:10:400
10195582252,6sleep00-21swapper/019:06:400
10195582252,6sleep00-21swapper/019:06:390
11109042240,2sleep013-21ksoftirqd/020:35:560
1020010992421,2cyclictest1300047-21fschecks_time23:50:432
1020010992421,2cyclictest1270395-21fschecks_count23:20:422
1020010992419,4cyclictest666912-21kworker/u8:4+flush-8:019:31:012
1020010992419,3cyclictest1194979-21unixbench_multi22:00:552
1020004992420,3cyclictest1290616-21tr23:40:450
1020004992420,3cyclictest1290616-21tr23:40:440
1020004992420,3cyclictest1225185-21kworker/u8:1+flush-8:023:55:580
1020004992420,3cyclictest1225185-21kworker/u8:1+flush-8:022:45:570
1020004992419,4cyclictest1171723-21irqrtprio21:40:440
1020004992419,3cyclictest1116873-21cpuspeed_turbos20:45:400
1020004992419,3cyclictest1116873-21cpuspeed_turbos20:45:390
1020004992419,3cyclictest0-21swapper/020:55:390
1020004992418,4cyclictest1343758-21unixbench_multi00:30:570
1020004992417,6cyclictest1076200-21kworker/u8:0+flush-8:022:35:510
102001099235,3cyclictest12684382kthreadcore23:15:472
1020010992320,2cyclictest1220901-21fschecks_time22:30:412
1020010992319,3cyclictest1052313-21cpuspeed_turbos19:40:392
1020010992319,2cyclictest1235699-21cpuspeed_turbos22:45:392
1020010992318,3cyclictest0-21swapper/222:20:002
1020009992314,6cyclictest0-21swapper/120:15:531
1020004992317,5cyclictest1076200-21kworker/u8:0+events_unbound22:00:500
1020004992315,6cyclictest1250348-21/usr/sbin/munin23:00:490
1020010992219,2cyclictest1101836-21aten2_r5power_p20:30:402
1020010992216,4cyclictest1280703-21irqrtprio23:30:462
1020010992213,7cyclictest1704890-21systemd-journal22:40:012
1020004992219,2cyclictest1019998-21cyclictest23:30:460
1020004992217,3cyclictest1279700-21kworker/u8:2+events_unbound00:20:400
1020004992217,3cyclictest1040975-21needreboot19:25:470
1020004992217,3cyclictest0-21swapper/021:50:500
1020004992215,6cyclictest1279303-21unixbench_singl23:25:580
102001099212,14cyclictest1175934-21apt-get21:45:392
102001099212,13cyclictest1061958-21apt-get19:50:392
1020010992117,3cyclictest1334111-21kworker/u8:1+flush-8:000:30:472
1020010992117,2cyclictest1156268-21aten2_r5power_v21:25:412
1020010992116,3cyclictest1024885-21turbostat.cron19:15:002
1020010992116,3cyclictest0-21swapper/200:35:562
1020010992114,5cyclictest1290606-21irqrtprio23:40:452
1020010992114,5cyclictest1290606-21irqrtprio23:40:452
1020004992117,3cyclictest1140687-21kworker/u8:2+flush-8:021:10:550
1020010992017,2cyclictest1230532-21aten2_r5power_c22:40:392
1020010992014,4cyclictest1076200-21kworker/u8:0+events_unbound20:35:012
1020010991915,3cyclictest1136679-21fschecks_count21:05:422
1020010991914,4cyclictest1304393-21kworker/u8:0+flush-8:023:59:032
1020010991911,6cyclictest1217746-21cut22:25:472
102001099191,13cyclictest1071867-21apt-get20:00:382
10152782193,9sleep30-21swapper/319:05:393
10152782193,9sleep30-21swapper/319:05:383
102001099182,8cyclictest0-21swapper/223:45:422
102001099182,11cyclictest1126500-21apt-get20:55:382
1020010991814,2cyclictest0-21swapper/221:10:452
1020010991813,3cyclictest1041934-21turbostat.cron19:30:012
1020010991810,6cyclictest1212175-21cut22:20:462
1020010991810,6cyclictest1212175-21cut22:20:462
1020010991810,6cyclictest1080506-21needreboot20:05:482
1020010991714,2cyclictest1161196-21aten2_r5power_p21:30:412
1020010991713,3cyclictest1151559-21fschecks_time21:20:452
1020010991713,2cyclictest0-21swapper/200:15:442
1020010991713,2cyclictest0-21swapper/200:10:582
1020010991712,3cyclictest1259388-21xargs23:05:572
1020010991712,3cyclictest0-21swapper/219:55:462
102001099171,11cyclictest1260241-21apt-get23:10:392
10713212168,5sleep30-21swapper/319:57:113
1020010991613,2cyclictest1186114-21fschecks_time21:55:412
1020010991613,2cyclictest1049091-21kthreadcore19:35:472
1020010991611,3cyclictest1131782-21fschecks_time21:00:422
1020010991611,3cyclictest0-21swapper/220:20:462
102001099156,7cyclictest1170740-21grep21:40:002
1020010991511,2cyclictest1180932-21aten2_r5power_v21:50:392
1020010991510,3cyclictest0-21swapper/223:00:472
12243312144,6sleep30-21swapper/322:30:513
102001099146,6cyclictest1057151-21cut19:45:392
1020010991410,2cyclictest0-21swapper/223:35:372
12759002133,6sleep30-21swapper/323:25:463
12599332132,7sleep30-21swapper/323:10:003
12441042132,7sleep30-21swapper/322:50:513
11757012132,6sleep30-21swapper/321:45:023
10962982132,7sleep30-21swapper/320:24:593
10765442132,6sleep30-21swapper/320:05:013
10624812133,6sleep30-21swapper/319:50:443
10308612132,7sleep30-21swapper/319:15:493
13446372122,6sleep30-21swapper/300:35:393
13380682123,5sleep30-21swapper/300:25:493
13242072122,5sleep30-21swapper/300:15:023
13199742122,6sleep30-21swapper/300:10:443
12907762122,7sleep30-21swapper/323:40:473
12907762122,7sleep30-21swapper/323:40:473
12701352122,6sleep30-21swapper/323:20:393
12673302122,6sleep30-21swapper/323:15:473
12512572122,6sleep30-21swapper/323:00:463
12350442122,6sleep30-21swapper/322:43:203
12252462121,6sleep30-21swapper/322:35:023
12144942120,2sleep3471rcuc/322:20:563
12144942120,2sleep3471rcuc/322:20:553
12059352122,6sleep30-21swapper/322:15:443
12005272122,6sleep30-21swapper/322:10:023
12005272122,6sleep30-21swapper/322:10:013
11960732122,6sleep30-21swapper/322:05:443
11953812122,6sleep30-21swapper/322:05:003
11866832122,6sleep30-21swapper/321:55:453
11854942122,5sleep30-21swapper/321:55:013
11515912122,6sleep30-21swapper/321:20:453
11416772122,6sleep30-21swapper/321:10:433
11351922122,6sleep30-21swapper/321:00:523
11303362122,6sleep30-21swapper/320:55:543
11253302122,6sleep30-21swapper/320:50:523
11157162122,6sleep30-21swapper/320:40:573
11017242122,6sleep30-21swapper/320:30:383
10832662123,6sleep30-21swapper/320:10:483
10719262122,6sleep30-21swapper/320:00:403
10607862122,6sleep30-21swapper/319:45:523
10518222122,6sleep30-21swapper/319:40:023
10415002122,6sleep30-21swapper/319:25:573
10363402122,6sleep30-21swapper/319:20:563
13145062112,5sleep30-21swapper/300:05:023
13100312112,5sleep30-21swapper/300:00:453
13083972112,5sleep30-21swapper/323:55:523
13002502111,6sleep30-21swapper/323:50:463
12947942112,6sleep30-21swapper/323:45:353
12846282112,5sleep30-21swapper/323:35:003
12843062112,5sleep30-21swapper/323:31:003
12453582112,6sleep30-21swapper/322:55:283
12390122112,5sleep30-21swapper/322:45:493
12193812112,5sleep30-21swapper/322:25:513
11709372112,6sleep30-21swapper/321:40:083
11659542112,5sleep30-21swapper/321:35:023
11636062112,5sleep30-21swapper/321:30:493
11564152112,5sleep30-21swapper/321:25:443
11462052112,6sleep30-21swapper/321:15:123
11399072112,5sleep30-21swapper/321:05:483
11170152112,5sleep30-21swapper/320:45:433
11170152112,5sleep30-21swapper/320:45:433
11107402112,6sleep30-21swapper/320:35:553
10905282112,5sleep30-21swapper/320:15:533
10421542112,5sleep30-21swapper/319:30:033
10201632113,5sleep30-21swapper/319:10:363
12602132102,5sleep30-21swapper/323:10:373
10965432102,5sleep30-21swapper/320:25:083
10470742102,5sleep30-21swapper/319:35:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional