You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 07:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Wed May 13, 2026 00:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
47441522070,5sleep375-21kworker/3:1-events19:30:013
5985702820,18sleep375-21kworker/3:1-events21:35:013
4930922760,4chrt0-21swapper/119:46:001
5533822710,3sleep375-21kworker/3:1-events20:46:123
452588996353,8cyclictest637483-21ntpq22:11:031
452588996157,3cyclictest598768-21kworker/u8:0+flush-8:023:16:121
452588996155,4cyclictest657992-21kworker/u8:3+events_unbound23:45:551
7531152590,5chrt0-21swapper/200:10:572
452588995955,3cyclictest703083-21kworker/u8:2+flush-8:000:31:111
452588995955,3cyclictest703083-21kworker/u8:2+flush-8:000:31:111
452584995746,9cyclictest499725-21diskstats19:55:550
452588995651,4cyclictest703083-21kworker/u8:2+events_unbound23:21:101
452588995549,4cyclictest598768-21kworker/u8:0+events_unbound22:25:551
452588995439,11cyclictest280-21systemd-journal23:52:181
452588995349,3cyclictest703083-21kworker/u8:2+events_unbound23:36:111
452588995349,3cyclictest504355-21kworker/u8:3+flush-8:021:56:101
452588995343,8cyclictest767213-21latency_hist00:24:591
452588995343,7cyclictest280-21systemd-journal22:43:181
452584995342,8cyclictest741540-21ntpq23:56:040
452588995248,3cyclictest682706-21kworker/u8:1+events_unbound23:01:101
452588995248,3cyclictest657992-21kworker/u8:3+flush-8:023:26:101
452588995248,3cyclictest583423-21kworker/u8:1+flush-8:021:30:581
452588995248,3cyclictest464573-21kworker/u8:1+flush-8:019:26:061
452588995243,7cyclictest280-21systemd-journal22:30:181
452584995244,5cyclictest577933-21ntpq21:11:040
452584995243,6cyclictest752392-21latency_hist00:10:010
6123812510,1sleep1612382-21cat21:46:011
452588995147,3cyclictest479013-21kworker/u8:2+flush-8:020:31:081
452588995146,4cyclictest703083-21kworker/u8:2+events_unbound00:06:151
452588995146,3cyclictest484340-21kworker/u8:1+events_unbound20:05:581
7099632500,3sleep0201rcuc/023:26:010
452588995046,3cyclictest703083-21kworker/u8:2+events_unbound23:41:101
452588995046,3cyclictest410867-21kworker/u8:3+events_unbound19:35:551
452588995044,5cyclictest632870-21kworker/u8:2+events_unbound22:35:531
452588995044,5cyclictest632870-21kworker/u8:2+events_unbound22:35:531
452588994945,3cyclictest703083-21kworker/u8:2+events_unbound00:26:111
452588994945,3cyclictest484340-21kworker/u8:1+events_unbound20:15:571
452588994944,4cyclictest504355-21kworker/u8:3+flush-8:022:05:581
452588994941,5cyclictest280-21systemd-journal20:01:181
452584994938,8cyclictest469280-21latency_hist19:25:010
7650672480,2sleep0201rcuc/000:21:020
452588994844,3cyclictest632870-21kworker/u8:2+flush-8:022:51:111
452584994842,5cyclictest464573-21kworker/u8:1+flush-8:019:26:100
452588994743,3cyclictest657992-21kworker/u8:3+events_unbound22:46:091
452588994743,3cyclictest598768-21kworker/u8:0+events_unbound22:56:101
452588994741,4cyclictest579119-21apt-get21:15:551
452584994740,5cyclictest280-21systemd-journal20:41:190
452588994639,5cyclictest737743-21kworker/u8:1+events_unbound00:00:521
452588994633,9cyclictest280-21systemd-journal22:16:171
452588994541,3cyclictest538715-21kworker/u8:0+flush-8:020:40:541
452588994540,4cyclictest657992-21kworker/u8:3+events_unbound23:11:111
452588994539,4cyclictest713061-21apt-get23:30:521
452588994536,6cyclictest518894-21latency_hist20:15:011
452588994536,6cyclictest518894-21latency_hist20:15:011
452588994440,3cyclictest538715-21kworker/u8:0+flush-8:020:50:571
452588994339,3cyclictest484340-21kworker/u8:1+events_unbound20:35:571
452584994334,6cyclictest280-21systemd-journal23:34:590
452588994239,2cyclictest504355-21kworker/u8:3+events_unbound21:36:101
452584994234,6cyclictest523861-21apt-get20:20:000
452588994137,3cyclictest573529-21kworker/u8:2+flush-8:021:11:031
452584994132,6cyclictest638380-21latency_hist22:15:010
452584994131,8cyclictest594169-21diskstats21:30:570
452588994036,3cyclictest598768-21kworker/u8:0+flush-8:023:06:071
452588994035,4cyclictest583423-21kworker/u8:1+events_unbound22:20:561
452588994028,8cyclictest280-21systemd-journal21:01:191
452584994032,6cyclictest727905-21/usr/sbin/munin23:46:030
452584994032,5cyclictest633189-21latency_hist22:10:010
452584994031,6cyclictest553806-21latency_hist20:50:010
452584994028,10cyclictest643289-21apt-get22:20:000
452588993935,3cyclictest737743-21kworker/u8:1+flush-8:000:16:201
452588993935,3cyclictest737743-21kworker/u8:1+flush-8:000:11:071
452588993935,3cyclictest583423-21kworker/u8:1+flush-8:021:51:071
452588993934,4cyclictest504355-21kworker/u8:3+flush-8:021:06:181
452584993930,7cyclictest569469-21diskstats21:05:570
452584993929,7cyclictest538911-21latency_hist20:35:010
452588993834,3cyclictest421273-21kworker/u8:0+events_unbound19:55:201
452588993725,9cyclictest280-21systemd-journal19:55:001
452588993632,3cyclictest732755-21kworker/u8:0+events_unbound00:35:541
452588993632,3cyclictest549060-21kworker/u8:2+events_unbound20:45:191
452588993632,3cyclictest538715-21kworker/u8:0+events_unbound20:59:181
452588993632,3cyclictest464573-21kworker/u8:1+events_unbound19:21:151
452584993627,6cyclictest280-21systemd-journal21:55:170
452588993531,3cyclictest583423-21kworker/u8:1+events_unbound21:20:561
452588993531,3cyclictest504355-21kworker/u8:3+events_unbound20:27:571
452588993529,4cyclictest421273-21kworker/u8:0+flush-8:019:10:261
452588993525,7cyclictest488184-21ntpq19:41:031
452584993527,5cyclictest280-21systemd-journal22:20:030
452584993526,6cyclictest499164-21latency_hist19:55:010
452584993525,8cyclictest657462-21ntpq22:31:030
4511192353,27sleep10-21swapper/119:06:041
452592993426,6cyclictest280-21systemd-journal19:52:492
452592993328,3cyclictest644202-21fschecks_time22:20:562
452588993329,3cyclictest504355-21kworker/u8:3+flush-8:021:26:191
452588993328,4cyclictest452739-21kworker/u8:2+events_unbound19:15:181
452588993328,3cyclictest583423-21kworker/u8:1+events_unbound21:43:101
452584993329,3cyclictest737743-21kworker/u8:1+events_unbound00:16:140
452584993329,3cyclictest583423-21kworker/u8:1+events_unbound21:41:090
452584993325,6cyclictest648762-21/usr/sbin/munin22:26:040
452588993228,3cyclictest504355-21kworker/u8:3+events_unbound22:01:131
452588993228,3cyclictest479013-21kworker/u8:2+events_unbound19:34:191
452584993228,3cyclictest598768-21kworker/u8:0+flush-8:022:51:000
452584993226,4cyclictest504355-21kworker/u8:3+flush-8:022:01:090
452584993225,5cyclictest504355-21kworker/u8:3+flush-8:021:36:080
4522622324,9sleep20-21swapper/219:08:142
45259299311,28cyclictest778596-21kthreadcore00:36:002
452584993126,3cyclictest632870-21kworker/u8:2+flush-8:022:55:490
4521082313,3sleep00-21swapper/019:06:140
452592993023,5cyclictest702838-21awk23:20:002
452592993022,6cyclictest489183-21cp19:45:002
452592993021,7cyclictest688045-21cron23:05:002
452588993025,4cyclictest737743-21kworker/u8:1+events_unbound23:55:201
45258499307,17cyclictest583757-21kworker/0:2+events23:05:550
452584993027,1cyclictest583423-21kworker/u8:1+events_unbound21:46:090
452584993024,4cyclictest768011-21diskstats00:25:560
452584993024,4cyclictest528657-21grep20:21:120
452584993024,4cyclictest504355-21kworker/u8:3+events_unbound20:56:100
452584993022,6cyclictest703145-21/usr/sbin/munin23:21:040
452584993021,6cyclictest588687-21latency_hist21:25:010
452584992923,5cyclictest421273-21kworker/u8:0+flush-8:019:12:490
452584992920,6cyclictest632870-21kworker/u8:2+flush-8:022:41:060
452584992919,8cyclictest489088-21apt-get19:45:030
452592992825,2cyclictest743012-21cpuspeed_turbos00:00:542
452592992824,2cyclictest773032-21fschecks_count00:30:572
452592992824,2cyclictest773032-21fschecks_count00:30:572
452592992823,3cyclictest772078-21unixbench_singl00:26:132
452584992824,3cyclictest657992-21kworker/u8:3+flush-8:023:51:010
452584992823,4cyclictest421273-21kworker/u8:0+events_unbound19:15:040
452584992823,3cyclictest583361-21unixbench_singl21:16:150
452584992822,4cyclictest421273-21kworker/u8:0+events_unbound19:40:560
452592992723,2cyclictest464691-21aten2_r5power_c19:20:542
452592992722,3cyclictest587753-21grep21:21:042
452592992722,3cyclictest503743-21unixbench_singl19:56:122
452592992622,3cyclictest623274-21turbostat.cron22:00:002
452592992622,2cyclictest638033-21unixbench_singl22:11:112
452592992622,2cyclictest539451-21tr20:35:542
45259299260,7cyclictest626663-21kthreadcore22:01:012
452588992622,3cyclictest479013-21kworker/u8:2+events_unbound20:23:191
452584992620,4cyclictest703083-21kworker/u8:2+flush-8:023:36:060
452584992618,6cyclictest683660-21diskstats23:00:570
452592992521,2cyclictest509393-21aten2_r5power_c20:05:532
452592992518,5cyclictest598531-21cron21:34:592
452592992517,6cyclictest568675-21apt-get21:05:002
452584992522,2cyclictest885-21mta-sts-daemon20:36:100
452584992522,2cyclictest474881-21fschecks_time19:30:580
452584992521,3cyclictest682706-21kworker/u8:1+events_unbound23:11:010
452584992520,4cyclictest657992-21kworker/u8:3+flush-8:022:48:490
452584992520,3cyclictest756963-21unixbench_singl00:11:120
452592992421,2cyclictest762730-21aten2_r5power_v00:20:552
452592992421,2cyclictest554335-21cpuspeed_turbos20:50:552
452592992421,2cyclictest518541-21unixbench_singl20:11:112
452592992421,2cyclictest518541-21unixbench_singl20:11:102
452592992420,3cyclictest668702-21aten2_r5power_v22:45:552
452592992420,2cyclictest479549-21aten2_r5power_p19:35:542
452592992419,4cyclictest632782-21unixbench_multi22:06:122
452592992419,3cyclictest524645-21cpuspeed_turbos20:20:552
45259299241,10cyclictest4583242kthreadcore19:15:022
452584992419,3cyclictest743017-21cpuspeed_turbos00:00:540
452584992418,4cyclictest632870-21kworker/u8:2+flush-8:022:36:050
452584992418,4cyclictest632870-21kworker/u8:2+flush-8:022:36:050
452584992417,5cyclictest533074-21ntpq20:26:040
452584992416,6cyclictest777982-21tune2fs00:35:580
44779622413,6sleep30-21swapper/319:05:463
452592992319,2cyclictest678746-21fschecks_count22:55:572
452592992319,2cyclictest589034-21aten2_r5power_c21:25:532
452592992319,2cyclictest0-21swapper/220:20:002
452584992319,2cyclictest0-21swapper/020:05:000
452584992318,3cyclictest515197-21kernelversion20:10:590
452584992318,3cyclictest515197-21kernelversion20:10:580
452584992318,3cyclictest0-21swapper/021:29:590
452584992314,1cyclictest0-21swapper/021:50:520
6781732220,13chrt371rcuc/222:55:012
452592992218,2cyclictest718221-21grep23:35:532
452592992218,2cyclictest0-21swapper/222:36:022
452592992218,2cyclictest0-21swapper/222:36:012
452592992217,3cyclictest727219-21unixbench_multi23:41:122
45258499228,5cyclictest0-21swapper/020:06:000
45258499228,12cyclictest723298-21grep23:40:570
452584992218,3cyclictest557919-21kworker/u8:1+flush-8:020:51:100
452584992218,3cyclictest484127-21systemd-run19:40:010
452584992218,2cyclictest0-21swapper/021:05:010
452584992217,4cyclictest598768-21kworker/u8:0+flush-8:023:15:480
452584992217,3cyclictest772992-21diskstats00:30:570
452584992217,3cyclictest772992-21diskstats00:30:560
6448122210,2sleep3481ktimers/322:20:583
452592992118,2cyclictest703397-21cpuspeed_turbos23:20:542
452592992118,2cyclictest603173-21unixbench_singl21:36:132
452592992117,2cyclictest549225-21aten2_r5power_p20:45:562
452592992117,2cyclictest0-21swapper/222:35:002
452592992116,4cyclictest20977-21cron23:10:002
452592992115,4cyclictest717275-21unixbench_multi23:31:122
452592992115,4cyclictest616956-21kthreadcore21:51:032
452592992112,6cyclictest643709-21sed22:20:012
45259299202,13cyclictest603832-21apt-get21:40:542
452592992017,2cyclictest504694-21fschecks_count20:00:552
452592992016,2cyclictest0-21swapper/219:15:002
45259299192,12cyclictest751518-21ntpq00:06:092
452592991915,2cyclictest0-21swapper/223:56:032
452592991914,4cyclictest537967-21needreboot20:31:042
452592991911,6cyclictest477166-21cut19:31:032
6223632180,2sleep3481ktimers/321:56:033
452592991814,2cyclictest0-21swapper/220:56:052
452592991814,2cyclictest0-21swapper/219:26:002
45259299172,11cyclictest693298-21apt-get23:10:532
452592991713,2cyclictest0-21swapper/222:28:212
452592991713,2cyclictest0-21swapper/220:45:012
7226592167,5sleep30-21swapper/323:40:003
45259299166,7cyclictest569662-21if_enp4s0.221:05:592
45259299162,13cyclictest0-21swapper/221:19:572
45259299162,13cyclictest0-21swapper/219:46:192
452592991613,2cyclictest0-21swapper/223:46:392
452592991613,2cyclictest0-21swapper/222:41:382
452592991612,2cyclictest578952-21latency_hist21:15:012
452592991612,2cyclictest533622-21unixbench_singl20:26:122
452592991612,2cyclictest0-21swapper/223:26:042
45259299161,11cyclictest736646-21ntpq23:51:052
45259299152,12cyclictest0-21swapper/200:16:422
452592991510,3cyclictest0-21swapper/221:46:002
7822802132,6sleep30-21swapper/300:40:013
6979122132,6sleep30-21swapper/323:15:023
6882282132,6sleep30-21swapper/323:05:013
6830622132,7sleep30-21swapper/323:00:013
6432712132,7sleep30-21swapper/322:20:003
5738462132,6sleep30-21swapper/321:10:023
5281862132,7sleep30-21swapper/320:21:063
4841182132,6sleep30-21swapper/319:40:013
7768252122,6sleep30-21swapper/300:31:143
7768252122,6sleep30-21swapper/300:31:143
7676722122,6sleep30-21swapper/300:25:033
7629632122,6sleep30-21swapper/300:20:583
7474802122,5sleep30-21swapper/300:05:013
7378372122,6sleep30-21swapper/323:55:533
7325122122,6sleep30-21swapper/323:50:013
7169512122,6sleep30-21swapper/323:31:113
7125962122,6sleep30-21swapper/323:28:183
6738702122,6sleep30-21swapper/322:50:573
6725752122,6sleep30-21swapper/322:46:123
6573352122,6sleep30-21swapper/322:31:043
6331742122,5sleep30-21swapper/322:10:013
6234082122,5sleep30-21swapper/322:00:023
6084372122,5sleep30-21swapper/321:45:013
5985702122,6sleep30-21swapper/321:35:023
5793662122,6sleep30-21swapper/321:16:003
5600342122,5sleep30-21swapper/320:56:013
5576702122,6sleep30-21swapper/320:51:043
5445742122,6sleep30-21swapper/320:40:573
5197092122,6sleep30-21swapper/320:16:003
5178632122,6sleep30-21swapper/320:11:033
5178632122,6sleep30-21swapper/320:11:033
5082522122,6sleep30-21swapper/320:01:083
4998542122,6sleep30-21swapper/319:55:583
4794372122,6sleep30-21swapper/319:35:523
4669412122,6sleep30-21swapper/319:21:033
4574282122,5sleep30-21swapper/319:15:003
4529542122,6sleep30-21swapper/319:10:563
7575502113,5sleep30-21swapper/300:15:053
7572882113,5sleep30-21swapper/300:14:433
7427152112,5sleep30-21swapper/300:00:163
7340742112,5sleep30-21swapper/323:51:013
7188902112,5sleep30-21swapper/323:36:003
7064492112,5sleep30-21swapper/323:21:033
6935482112,6sleep30-21swapper/323:10:563
6870272112,5sleep30-21swapper/323:01:053
6656482112,5sleep30-21swapper/322:41:033
6583642112,5sleep30-21swapper/322:35:013
6583642112,5sleep30-21swapper/322:35:013
6486032112,5sleep30-21swapper/322:25:023
6285612112,5sleep30-21swapper/322:05:553
6136072112,5sleep30-21swapper/321:50:033
6109062112,5sleep30-21swapper/321:46:013
5889162112,5sleep30-21swapper/321:25:033
5842952112,5sleep30-21swapper/321:20:573
5691722112,5sleep30-21swapper/321:05:363
5639382112,5sleep30-21swapper/321:00:023
5428392112,5sleep30-21swapper/320:36:033
5345392112,5sleep30-21swapper/320:30:563
5293512112,5sleep30-21swapper/320:25:523
5090732111,5sleep30-21swapper/320:05:013
4944632112,6sleep30-21swapper/319:50:123
4899572112,5sleep30-21swapper/319:45:543
4744152112,5sleep30-21swapper/319:30:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional