You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-18 - 10:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Fri Sep 18, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21524021210,3sleep10-21swapper/122:44:091
16182750,7chrt0-21swapper/119:09:111
2713995644,9cyclictest278-21systemd-journal20:20:000
2713995244,6cyclictest242769-21latency_hist23:10:010
1266412510,1sleep0126642-21grep21:14:080
2718995039,8cyclictest273038-21latency_hist23:40:011
2713994945,3cyclictest3351787-21kworker/u8:4+flush-8:019:20:010
2713994740,5cyclictest133739-21latency_hist21:20:010
2718994541,2cyclictest168435-21idleruntime-cro21:55:001
2713994534,8cyclictest203800-21/usr/sbin/munin22:34:080
1797062450,6sleep20-21swapper/222:09:082
2718994336,5cyclictest3351787-21kworker/u8:4+flush-8:019:14:031
2713994336,5cyclictest76828-21kworker/u8:2+flush-8:020:49:090
2713994332,8cyclictest262840-21apt-get23:30:010
2713994332,8cyclictest152904-21ntpq21:39:100
2713994236,4cyclictest257804-21kworker/u8:0+flush-8:000:39:020
2718994134,5cyclictest139267-21apt-get21:28:581
2713994135,4cyclictest76828-21kworker/u8:2+flush-8:020:39:150
2713994135,4cyclictest76828-21kworker/u8:2+flush-8:020:39:150
2720994022,16cyclictest278-21systemd-journal21:39:372
2718994033,5cyclictest49899-21apt-get19:58:581
2718994030,7cyclictest25180-21/usr/sbin/munin19:34:111
2718994029,8cyclictest278-21systemd-journal22:30:371
2713994036,3cyclictest187989-21kworker/u8:1+events_unbound23:34:070
2713994031,7cyclictest243572-21diskstats23:14:020
2718993932,5cyclictest113047-21ntpq20:59:111
2713993921,9cyclictest278-21systemd-journal21:52:370
2720993822,13cyclictest54169-21unixbench_singl19:59:182
2718993833,3cyclictest54529-21kworker/u8:1+events_unbound20:29:061
2720993721,14cyclictest292979-21unixbench_singl23:59:212
2718993729,6cyclictest88960-21latency_hist20:35:001
2713993727,7cyclictest144178-21/usr/sbin/munin21:34:090
2720993627,7cyclictest124042-21cat21:10:012
2718993627,6cyclictest278-21systemd-journal19:35:371
2718993626,7cyclictest158548-21latency_hist21:45:001
2718993626,7cyclictest158548-21latency_hist21:45:001
2713993629,4cyclictest257698-21grep23:24:180
2713993625,8cyclictest284032-21diskstats23:54:020
2720993521,12cyclictest248316-21aten2_r5power_c23:19:002
2718993528,5cyclictest278-21systemd-journal00:20:001
2718993526,7cyclictest168965-21/usr/sbin/munin21:59:101
2718993526,7cyclictest159074-21/usr/sbin/munin21:49:181
2718993526,6cyclictest39500-21latency_hist19:45:011
2713993531,3cyclictest4142839-21kworker/u8:0+events_unbound19:44:060
2713993530,3cyclictest2418-21kworker/u8:2+flush-8:020:09:040
2713993525,7cyclictest223004-21latency_hist22:50:000
2713993525,7cyclictest223004-21latency_hist22:50:000
41919672344,9sleep20-21swapper/219:08:402
2720993428,4cyclictest293336-21kworker/u8:1+flush-8:000:22:562
2720993420,12cyclictest253529-21aten2_r5power_e23:24:002
2718993430,3cyclictest287554-21kworker/u8:2+flush-8:000:05:001
2718993424,8cyclictest252665-21latency_hist23:20:011
2713993424,8cyclictest162823-21ntpq21:49:130
2720993322,8cyclictest278-21systemd-journal22:15:002
2720993319,12cyclictest0-21swapper/223:10:002
2718993327,4cyclictest76828-21kworker/u8:2+flush-8:020:54:091
2713993325,6cyclictest323524-21/usr/sbin/munin00:34:180
2713993322,9cyclictest0-21swapper/023:20:000
2713993322,8cyclictest44412-21latency_hist19:50:010
272099323,26cyclictest156031-21cut21:44:072
272099323,26cyclictest156031-21cut21:44:072
2718993229,2cyclictest254395-21kthreadcore23:24:061
2718993228,2cyclictest123643-21systemd-run21:10:011
2713993227,3cyclictest390-21dbus-daemon23:00:000
2713993226,4cyclictest257804-21kworker/u8:0+flush-8:000:20:020
272099313,15cyclictest0-21swapper/221:54:012
2720993121,8cyclictest278-21systemd-journal19:20:002
272099311,23cyclictest94500-21apt-get20:44:002
272099311,23cyclictest268609-21apt-get23:39:002
272099311,23cyclictest258410-21apt-get23:29:002
2718993125,5cyclictest64186-21kworker/u8:3+flush-8:020:44:041
2713993126,4cyclictest3351787-21kworker/u8:4+events_unbound19:59:150
2713993126,3cyclictest237858-21systemd-run23:05:010
2713993125,3cyclictest118657-21cron21:05:000
2713993122,7cyclictest173376-21latency_hist22:00:010
2713993117,10cyclictest123600-21apt-get21:10:010
898682300,6sleep1281rcuc/120:39:021
898682300,6sleep1281rcuc/120:39:021
272099304,22cyclictest15349-21cut19:24:042
2720993022,6cyclictest278-21systemd-journal23:35:002
2720993016,12cyclictest0-21swapper/219:45:012
272099301,25cyclictest203823-21/usr/sbin/munin22:33:562
272099300,27cyclictest192346-21/usr/sbin/munin22:19:092
272099300,26cyclictest40098-21/usr/sbin/munin19:49:002
272099300,25cyclictest294193-21ps00:04:072
2718993024,4cyclictest148632-21latency_hist21:35:001
2718993023,5cyclictest2418-21kworker/u8:2+flush-8:020:04:151
2718993021,7cyclictest278-21systemd-journal22:54:591
2713993026,3cyclictest187989-21kworker/u8:1+events_unbound22:54:140
2713993025,4cyclictest76828-21kworker/u8:2+flush-8:022:21:580
2713993021,6cyclictest113714-21latency_hist21:00:010
2713993020,7cyclictest134282-21/usr/sbin/munin21:24:170
2720992924,3cyclictest322835-21unixbench_multi00:29:212
272099291,25cyclictest162958-21/usr/sbin/munin21:49:172
272099291,23cyclictest193948-21taskset22:24:012
272099291,23cyclictest134307-21apt-get21:23:592
272099291,22cyclictest283782-21apt-get23:53:592
272099290,25cyclictest228584-21/usr/sbin/munin22:59:022
272099290,17cyclictest78604-21/usr/sbin/munin20:24:162
2718992925,3cyclictest3351787-21kworker/u8:4+flush-8:019:24:031
2718992925,3cyclictest257804-21kworker/u8:0+flush-8:023:30:071
2718992924,4cyclictest287554-21kworker/u8:2+flush-8:000:39:151
2718992924,3cyclictest307855-21unixbench_singl00:14:211
2713992924,3cyclictest282537-21needreboot23:49:110
2713992923,4cyclictest88880-21unixbench_singl20:34:180
2713992923,4cyclictest287554-21kworker/u8:2+events_unbound00:14:160
2713992923,4cyclictest267420-21kworker/u8:2+events_unbound23:44:010
2713992922,5cyclictest3351787-21kworker/u8:4+events_unbound19:24:010
2713992922,5cyclictest272427-21ntpq23:39:130
2713992922,5cyclictest187989-21kworker/u8:1+events_unbound22:44:070
2713992922,4cyclictest54529-21kworker/u8:1+events_unbound20:14:020
2713992919,8cyclictest177744-21ntpq22:04:150
272099283,21cyclictest32152-21kthreadcore19:39:082
2720992824,2cyclictest198946-21aten2_r5power_c22:29:012
2720992814,12cyclictest0-21swapper/219:30:012
2720992813,13cyclictest84674-21aten2_r5power_v20:34:012
2720992813,12cyclictest0-21swapper/222:39:142
272099281,23cyclictest70080-21/usr/sbin/munin20:19:032
272099281,23cyclictest243355-21/usr/sbin/munin23:14:002
272099280,9cyclictest3242782tr00:34:072
272099280,25cyclictest110979-21tr20:59:082
272099280,24cyclictest89438-21taskset20:38:132
272099280,24cyclictest89438-21taskset20:38:132
272099280,24cyclictest64702-21/usr/sbin/munin20:13:592
272099280,24cyclictest58052kthreadcore19:14:082
272099280,24cyclictest233282-21seq23:02:062
272099280,24cyclictest143891-21seq21:30:172
2718992824,2cyclictest64170-21turbostat.cron20:10:011
2718992822,4cyclictest278-21systemd-journal23:00:001
2718992821,5cyclictest178906-21apt-get22:08:591
2718992820,6cyclictest128793-21latency_hist21:15:011
2713992823,4cyclictest4142839-21kworker/u8:0+flush-8:019:39:130
2713992822,4cyclictest257804-21kworker/u8:0+flush-8:023:57:120
2713992821,5cyclictest98867-21cron20:44:590
2713992818,8cyclictest23797-21ntpq19:29:130
475602270,3sleep1281rcuc/119:54:091
272099279,15cyclictest57984-21tr20:04:092
272099278,16cyclictest216903-21tr22:44:112
272099272,20cyclictest108139-21ntpq20:54:132
272099271,23cyclictest63446-21/usr/sbin/munin20:09:112
272099271,22cyclictest223538-21munin-node22:53:592
272099271,21cyclictest172874-21ps21:59:152
2720992712,13cyclictest0-21swapper/223:45:002
272099270,24cyclictest303714-21/usr/sbin/munin00:14:032
272099270,23cyclictest174191-21/usr/sbin/munin22:04:032
272099270,23cyclictest114532-21/usr/sbin/munin21:04:022
2718992724,2cyclictest0-21swapper/122:14:121
2718992723,3cyclictest323017-21systemd-run00:30:011
2718992723,2cyclictest0-21swapper/122:40:001
2718992721,3cyclictest0-21swapper/121:19:161
2718992720,5cyclictest40054-21apt-get19:48:581
2713992722,3cyclictest199445-21sort22:29:060
2713992721,5cyclictest76828-21kworker/u8:2+flush-8:020:28:570
2713992721,4cyclictest257804-21kworker/u8:0+flush-8:000:29:090
272099263,21cyclictest83916-21grep20:29:182
272099261,23cyclictest124847-21/usr/sbin/munin21:14:052
272099260,24cyclictest222485-21/usr/sbin/munin22:49:172
272099260,24cyclictest222485-21/usr/sbin/munin22:49:172
272099260,23cyclictest99357-21taskset20:48:172
272099260,23cyclictest308204-21taskset00:15:382
272099260,23cyclictest25798-21sed19:34:062
272099260,22cyclictest328323-21runrttasks00:37:312
272099260,22cyclictest129352-21/usr/sbin/munin21:18:592
2718992619,5cyclictest293336-21kworker/u8:1+flush-8:000:20:311
2713992621,3cyclictest45197-21tr19:54:010
272099250,23cyclictest278492-21taskset23:45:182
272099250,23cyclictest139059-21taskset21:26:362
272099250,22cyclictest301169-21cut00:09:122
272099250,22cyclictest301169-21cut00:09:122
2718992521,2cyclictest0-21swapper/123:44:021
2718992520,4cyclictest257804-21kworker/u8:0+flush-8:023:59:051
2718992520,4cyclictest187989-21kworker/u8:1+events_unbound23:29:021
2718992518,6cyclictest193073-21kworker/u8:0+events_unbound22:20:011
2718992517,6cyclictest233428-21/usr/sbin/munin23:04:111
2718992516,7cyclictest193252-21cp22:20:001
271399254,6cyclictest0-21swapper/000:04:000
271399250,22cyclictest183685-21sshd22:12:370
272099240,21cyclictest44909-21taskset19:53:362
2718992420,2cyclictest0-21swapper/119:15:031
2718992419,4cyclictest3351787-21kworker/u8:4+flush-8:019:29:181
2718992419,3cyclictest177959-21mailstats22:04:181
271399246,14cyclictest14553-21kworker/0:0+events00:06:030
271399246,14cyclictest14553-21kworker/0:0+events00:06:030
271399242,21cyclictest14553-21kworker/0:0+events21:40:350
271399242,21cyclictest14553-21kworker/0:0+events21:40:350
402342230,4sleep375-21kworker/3:1-events19:49:013
271899239,5cyclictest0-21swapper/121:22:421
2718992317,4cyclictest0-21swapper/121:04:021
2718992317,3cyclictest0-21swapper/123:48:591
2713992310,4cyclictest0-21swapper/020:22:070
271399230,22cyclictest14553-21kworker/0:0+events19:34:080
2718992218,2cyclictest285930-21kthreadcore23:54:071
2718992218,2cyclictest0-21swapper/120:49:071
2718992218,2cyclictest0-21swapper/100:09:191
2718992218,2cyclictest0-21swapper/100:09:191
2718992216,5cyclictest193073-21kworker/u8:0+flush-8:022:28:081
271399225,16cyclictest14553-21kworker/0:0+events22:09:110
271399223,18cyclictest14553-21kworker/0:0+events20:03:030
2713992219,2cyclictest839-21mta-sts-daemon22:40:000
2713992217,3cyclictest312702-21unixbench_multi00:19:220
2713992213,5cyclictest0-21swapper/019:13:590
271399220,21cyclictest14553-21kworker/0:0+events22:16:000
20052222,14sleep02209-21tail19:09:190
19322228,7sleep30-21swapper/319:09:183
2718992117,3cyclictest219375-21irqrtprio22:49:081
2718992117,3cyclictest219375-21irqrtprio22:49:081
2718992117,2cyclictest0-21swapper/120:20:001
2718992116,4cyclictest257804-21kworker/u8:0+events_unbound00:34:121
2718992115,4cyclictest247721-21idleruntime-cro23:15:001
2713992110,9cyclictest104642-21diskstats20:54:030
11410422113,5sleep30-21swapper/321:02:013
2718992016,2cyclictest0-21swapper/120:24:101
2718992016,2cyclictest0-21swapper/120:14:061
2718992012,6cyclictest238389-21apt-get23:09:001
2718992012,6cyclictest128813-21kworker/u8:1+flush-8:021:39:101
2713992016,2cyclictest0-21swapper/021:29:100
2686122172,9sleep30-21swapper/323:39:013
350992144,6sleep30-21swapper/319:44:003
2832152142,8sleep30-21swapper/323:50:003
2730782132,6sleep30-21swapper/323:40:013
2180832132,6sleep30-21swapper/322:45:003
2082562133,6sleep30-21swapper/322:35:013
997222122,6sleep30-21swapper/320:49:053
950012122,6sleep30-21swapper/320:44:073
81382122,6sleep30-21swapper/319:15:043
790942122,6sleep30-21swapper/320:25:013
733812122,6sleep30-21swapper/320:19:113
640472122,6sleep30-21swapper/320:09:203
590422122,6sleep30-21swapper/320:04:193
536802122,6sleep30-21swapper/319:59:143
343502122,6sleep30-21swapper/319:39:193
3304552121,6sleep30-21swapper/300:39:083
3194472122,6sleep30-21swapper/300:29:103
3164462122,6sleep30-21swapper/300:24:113
2981142122,6sleep30-21swapper/300:05:013
2981142122,6sleep30-21swapper/300:05:013
2936672122,6sleep30-21swapper/300:04:013
2835752122,6sleep30-21swapper/323:51:413
2581982122,6sleep30-21swapper/323:26:373
2572392122,6sleep30-21swapper/323:24:123
249502122,6sleep30-21swapper/319:31:043
2485342122,6sleep30-21swapper/323:19:033
2269222122,6sleep30-21swapper/322:54:113
2037992122,6sleep30-21swapper/322:33:573
1887582122,6sleep30-21swapper/322:19:003
1840642122,6sleep30-21swapper/322:14:033
1826132122,6sleep30-21swapper/322:09:133
1640152122,6sleep30-21swapper/321:54:013
1630752122,6sleep30-21swapper/321:49:193
145392122,6sleep30-21swapper/319:20:013
1436722122,6sleep30-21swapper/321:30:013
1430202123,5sleep30-21swapper/321:29:143
1191912122,6sleep30-21swapper/321:08:593
892322112,6sleep30-21swapper/320:35:363
892322112,6sleep30-21swapper/320:35:363
849332112,5sleep30-21swapper/320:34:043
745462112,6sleep30-21swapper/320:23:053
645482113,5sleep30-21swapper/320:12:023
3235822112,5sleep30-21swapper/300:34:023
3082272112,6sleep30-21swapper/300:15:563
3032852113,5sleep30-21swapper/300:10:513
30262112,6sleep30-21swapper/319:11:423
2885402112,6sleep30-21swapper/323:57:193
2633682112,6sleep30-21swapper/323:31:033
2431252112,6sleep30-21swapper/323:11:453
2381752112,6sleep30-21swapper/323:06:353
2331642112,6sleep30-21swapper/323:00:373
2294192112,5sleep30-21swapper/322:59:093
2184432112,6sleep30-21swapper/322:46:373
2184432112,6sleep30-21swapper/322:46:373
1986492112,6sleep30-21swapper/322:25:543
196872112,6sleep30-21swapper/319:25:113
1936152113,5sleep30-21swapper/322:20:053
1736462112,6sleep30-21swapper/322:00:273
1692462112,5sleep30-21swapper/321:59:023
1539172112,6sleep30-21swapper/321:41:253
1539172112,6sleep30-21swapper/321:41:253
1489922112,6sleep30-21swapper/321:36:473
1341802112,6sleep30-21swapper/321:22:393
1291032112,6sleep30-21swapper/321:16:093
1256352112,5sleep30-21swapper/321:14:083
1090782112,6sleep30-21swapper/320:56:073
1040732112,6sleep30-21swapper/320:50:133
446522102,5sleep30-21swapper/319:50:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional