You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-23 - 03:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Tue Dec 23, 2025 00:44:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2772132610,3chrt0-21swapper/022:44:010
65919995447,5cyclictest389499-21latency_hist00:35:010
65922995346,6cyclictest320449-21kworker/u8:4+flush-8:023:44:121
65922995247,4cyclictest251149-21kworker/u8:3+events_unbound22:55:001
65919995145,5cyclictest2874338-21kworker/u8:0+events_unbound23:05:020
65922995045,4cyclictest2874338-21kworker/u8:0+events_unbound23:00:001
65922995045,3cyclictest354638-21kworker/u8:1+flush-8:000:29:131
65922995044,4cyclictest166758-21kworker/u8:1+events_unbound20:50:001
65922994945,3cyclictest147275-21kworker/u8:1+flush-8:020:34:111
65922994944,4cyclictest251149-21kworker/u8:3+events_unbound22:35:011
65924994738,6cyclictest278-21systemd-journal19:45:012
65922994737,7cyclictest127361-21latency_hist20:10:001
65919994744,2cyclictest2874338-21kworker/u8:0+events_unbound20:44:110
65919994744,2cyclictest201809-21kworker/u8:2+events_unbound21:34:100
65922994642,3cyclictest2874338-21kworker/u8:0+flush-8:020:38:571
65919994642,3cyclictest251149-21kworker/u8:3+events_unbound22:54:110
65922994539,4cyclictest186699-21kworker/u8:3+events_unbound21:14:091
65922994535,7cyclictest374699-21latency_hist00:20:001
65924994439,4cyclictest2874338-21kworker/u8:0+flush-8:019:39:112
65922994437,6cyclictest2874338-21kworker/u8:0+events_unbound19:35:021
65919994440,3cyclictest320449-21kworker/u8:4+events_unbound23:39:100
65922994339,3cyclictest2874338-21kworker/u8:0+flush-8:021:33:581
65922994337,5cyclictest2874338-21kworker/u8:0+events_unbound23:03:561
65922994337,5cyclictest251149-21kworker/u8:3+flush-8:023:59:131
65922994337,5cyclictest251149-21kworker/u8:3+events_unbound23:38:561
65922994337,5cyclictest251149-21kworker/u8:3+events_unbound00:08:561
65922994337,4cyclictest315456-21kworker/u8:1+events_unbound23:24:131
65919994339,3cyclictest2874338-21kworker/u8:0+events_unbound20:54:110
65924994236,4cyclictest251149-21kworker/u8:3+flush-8:023:28:562
65922994237,4cyclictest251149-21kworker/u8:3+events_unbound23:28:551
65919994239,2cyclictest87714-21kworker/u8:3+events_unbound19:30:370
65919994239,2cyclictest370024-21kworker/u8:2+events_unbound00:20:120
65919994238,3cyclictest2874338-21kworker/u8:0+events_unbound20:04:010
65922994136,4cyclictest142056-21kworker/u8:2+events_unbound20:43:561
65922994136,3cyclictest2874338-21kworker/u8:0+flush-8:019:44:101
65922994136,3cyclictest2874338-21kworker/u8:0+events_unbound22:04:131
65922994136,3cyclictest270929-21kworker/u8:1+events_unbound22:49:151
65922994135,5cyclictest251149-21kworker/u8:3+flush-8:023:19:141
65919994137,3cyclictest2874338-21kworker/u8:0+events_unbound21:13:550
2392932410,3sleep30-21swapper/322:04:043
65922994036,3cyclictest201809-21kworker/u8:2+events_unbound21:44:071
65922994035,3cyclictest2874338-21kworker/u8:0+events_unbound22:24:001
65922994035,3cyclictest201809-21kworker/u8:2+events_unbound22:13:591
65922994034,5cyclictest2874338-21kworker/u8:0+events_unbound19:58:581
65922994034,5cyclictest285451-21kworker/u8:2+events_unbound23:49:071
65922994033,5cyclictest251149-21kworker/u8:3+flush-8:000:04:131
65919994036,3cyclictest251149-21kworker/u8:3+events_unbound00:04:140
65919994032,3cyclictest142056-21kworker/u8:2+events_unbound21:18:590
65922993934,3cyclictest2874338-21kworker/u8:0+events_unbound19:29:121
65922993933,5cyclictest285451-21kworker/u8:2+events_unbound23:53:581
65919993935,3cyclictest2874338-21kworker/u8:0+events_unbound22:29:310
65919993935,3cyclictest2874338-21kworker/u8:0+events_unbound19:38:570
65919993935,3cyclictest270929-21kworker/u8:1+events_unbound22:45:120
65919993935,3cyclictest251149-21kworker/u8:3+events_unbound23:30:140
65922993834,3cyclictest3059834-21kworker/u8:3+events_unbound19:19:121
65922993832,5cyclictest370024-21kworker/u8:2+flush-8:000:24:141
65919993836,1cyclictest201rcuc/020:39:110
65919993834,3cyclictest2874338-21kworker/u8:0+events_unbound21:23:550
65919993834,3cyclictest251149-21kworker/u8:3+events_unbound22:59:130
65919993831,3cyclictest365074-21kworker/u8:0+events_unbound00:14:150
65922993731,5cyclictest211398-21kworker/u8:1+flush-8:021:54:131
65922993730,5cyclictest231545-21apt-get21:57:101
65922993729,6cyclictest137726-21/usr/sbin/munin20:24:041
65919993733,3cyclictest384499-21kworker/u8:3+events_unbound00:35:120
65919993733,3cyclictest370024-21kworker/u8:2+events_unbound00:25:020
65919993733,3cyclictest251149-21kworker/u8:3+events_unbound23:55:390
65919993733,3cyclictest152234-21kworker/u8:4+events_unbound20:45:130
65919993726,9cyclictest181619-21latency_hist21:05:000
65919993714,21cyclictest576-21runrttasks23:25:380
65922993630,5cyclictest389846-21kworker/u8:2+events_unbound00:39:171
65919993633,2cyclictest87714-21kworker/u8:3+events_unbound19:47:110
65919993633,2cyclictest2874338-21kworker/u8:0+events_unbound22:22:110
65919993632,3cyclictest2874338-21kworker/u8:0+events_unbound22:09:300
65919993632,3cyclictest2874338-21kworker/u8:0+events_unbound20:09:300
65919993632,3cyclictest2874338-21kworker/u8:0+events_unbound19:13:550
65919993632,3cyclictest251149-21kworker/u8:3+events_unbound23:17:130
65919993632,3cyclictest251149-21kworker/u8:3+events_unbound23:14:140
65919993632,3cyclictest251149-21kworker/u8:3+events_unbound22:15:360
65919993632,3cyclictest142056-21kworker/u8:2+events_unbound20:30:370
6591999362,31cyclictest411-21dbus-daemon20:15:010
65922993531,3cyclictest87714-21kworker/u8:3+flush-8:019:48:561
65922993531,3cyclictest166758-21kworker/u8:1+flush-8:020:58:581
65922993526,3cyclictest2874338-21kworker/u8:0+events_unbound21:24:091
65922993524,5cyclictest201809-21kworker/u8:2+events_unbound21:39:041
65919993531,3cyclictest2874338-21kworker/u8:0+events_unbound22:33:320
65919993531,3cyclictest2874338-21kworker/u8:0+events_unbound22:10:360
65919993531,3cyclictest2874338-21kworker/u8:0+events_unbound22:04:050
65919993531,3cyclictest285451-21kworker/u8:2+events_unbound23:52:210
65919993531,3cyclictest251149-21kworker/u8:3+events_unbound00:07:250
65919993531,3cyclictest102960-21kworker/u8:1+events_unbound20:24:150
65919993530,4cyclictest2874338-21kworker/u8:0+events_unbound19:19:060
65919993530,3cyclictest320449-21kworker/u8:4+events_unbound23:41:140
65919993526,7cyclictest182650-21diskstats21:08:560
65919993526,7cyclictest182650-21diskstats21:08:560
65922993429,4cyclictest87714-21kworker/u8:3+flush-8:020:19:111
65922993429,3cyclictest166758-21kworker/u8:1+events_unbound20:53:591
65919993431,2cyclictest2874338-21kworker/u8:0+events_unbound20:57:110
65919993430,3cyclictest251149-21kworker/u8:3+events_unbound22:35:110
65922993328,4cyclictest201809-21kworker/u8:2+events_unbound21:49:101
65919993330,2cyclictest166758-21kworker/u8:1+events_unbound21:26:440
65919993329,3cyclictest354638-21kworker/u8:1+events_unbound00:18:560
65919993329,3cyclictest211398-21kworker/u8:1+events_unbound21:45:250
65919993328,3cyclictest201809-21kworker/u8:2+events_unbound21:40:360
65919993323,8cyclictest146452-21ntpq20:29:060
3081322330,4sleep10-21swapper/123:14:031
65924993223,7cyclictest278-21systemd-journal22:34:112
65922993227,4cyclictest2874338-21kworker/u8:0+flush-8:020:04:081
65922993226,4cyclictest2874338-21kworker/u8:0+events_unbound19:13:571
65922993226,4cyclictest275447-21grep22:39:111
65919993228,3cyclictest87714-21kworker/u8:3+events_unbound19:54:550
65919993228,3cyclictest87714-21kworker/u8:3+events_unbound19:43:120
65919993228,3cyclictest285451-21kworker/u8:2+events_unbound23:49:150
65919993228,3cyclictest285451-21kworker/u8:2+events_unbound23:24:230
65919993228,3cyclictest24411-21kworker/u8:1+events_unbound19:22:100
65919993228,3cyclictest201809-21kworker/u8:2+events_unbound21:53:110
1421722320,3chrt575-21kworker/3:2+events20:25:013
65922993125,5cyclictest320449-21kworker/u8:4+events_unbound00:14:061
65922993125,4cyclictest255860-21latency_hist22:20:011
65919993127,3cyclictest231386-21kworker/u8:3+events_unbound21:55:370
65919993121,4cyclictest211398-21kworker/u8:1+events_unbound21:37:090
65924993024,4cyclictest191508-21unixbench_multi21:14:122
65922993027,2cyclictest192951-21irqrtprio21:19:001
65922993026,3cyclictest2874338-21kworker/u8:0+flush-8:022:05:251
65922993026,3cyclictest2874338-21kworker/u8:0+events_unbound22:29:141
65922993025,4cyclictest2874338-21kworker/u8:0+events_unbound21:04:091
65919993026,3cyclictest87714-21kworker/u8:3+events_unbound19:55:010
65919993026,3cyclictest2874338-21kworker/u8:0+events_unbound19:29:290
65919993020,8cyclictest136542-21ntpq20:19:050
611152303,8sleep20-21swapper/219:05:322
65924992924,3cyclictest374527-21unixbench_multi00:19:132
65924992923,4cyclictest300101-21unixbench_singl23:04:132
65924992823,4cyclictest2874338-21kworker/u8:0+flush-8:020:18:572
65922992824,3cyclictest251149-21kworker/u8:3+flush-8:023:09:101
65922992821,6cyclictest142056-21kworker/u8:2+flush-8:020:29:031
65919992824,3cyclictest2874338-21kworker/u8:0+events_unbound23:01:120
65924992723,3cyclictest128220-21fschecks_time20:13:582
65922992723,3cyclictest365074-21kworker/u8:0+events_unbound00:33:311
65922992723,3cyclictest166758-21kworker/u8:1+flush-8:021:28:581
65922992721,5cyclictest24411-21kworker/u8:1+events_unbound19:23:591
65922992720,5cyclictest2874338-21kworker/u8:0+events_unbound20:14:101
65922992719,6cyclictest108237-21apt-get19:53:531
6130322716,6sleep30-21swapper/319:07:583
65922992622,3cyclictest2874338-21kworker/u8:0+flush-8:019:34:131
65922992622,3cyclictest2874338-21kworker/u8:0+events_unbound21:08:591
65922992622,3cyclictest2874338-21kworker/u8:0+events_unbound21:08:591
65922992622,3cyclictest270929-21kworker/u8:1+events_unbound22:43:561
65924992522,2cyclictest66399-21aten2_r5power_e19:13:562
65924992522,2cyclictest389490-21cron00:35:002
65924992522,2cyclictest325940-21fschecks_count23:33:582
65924992522,2cyclictest152599-21aten2_r5power_e20:38:542
65924992522,2cyclictest127203-21unixbench_multi20:09:132
65924992522,2cyclictest0-21swapper/222:20:002
65924992521,2cyclictest369768-21cron00:15:002
65924992520,3cyclictest390978-21kthreadcore00:39:042
65922992521,2cyclictest0-21swapper/123:34:111
612902252,6sleep00-21swapper/019:07:480
65924992420,2cyclictest212213-21cpuspeed_turbos21:38:552
65924992420,2cyclictest0-21swapper/219:35:002
65924992419,4cyclictest354685-21idleruntime-cro00:00:002
612812244,14sleep10-21swapper/119:07:401
65924992320,2cyclictest310982-21aten2_r5power_v23:18:582
65924992320,2cyclictest162741-21fschecks_count20:48:562
65924992319,2cyclictest375617-21fschecks_time00:23:592
65924992319,2cyclictest364506-21grep00:09:122
65924992318,3cyclictest274906-21taskset22:39:022
65924992318,3cyclictest0-21swapper/222:30:002
65924992219,2cyclictest137864-21aten2_r5power_v20:23:542
65924992218,3cyclictest142056-21kworker/u8:2+flush-8:020:28:592
65924992218,2cyclictest87472-21cron19:30:002
65924992218,2cyclictest172386-21aten2_r5power_p20:58:562
65924992118,2cyclictest295143-21unixbench_multi22:59:132
65924992118,2cyclictest285451-21kworker/u8:2+flush-8:023:53:592
65924992118,2cyclictest202180-21aten2_r5power_e21:28:552
65924992117,3cyclictest359678-21cron00:05:002
65924992117,2cyclictest157736-21cpuspeed_turbos20:43:562
65924992114,5cyclictest181622-21apt-get21:05:002
65924992110,4cyclictest0-21swapper/222:09:132
6592499208,4cyclictest0-21swapper/223:23:082
6592499208,4cyclictest0-21swapper/221:49:152
6592499207,5cyclictest0-21swapper/223:05:032
65924992017,2cyclictest236769-21aten2_r5power_e22:03:562
65924992016,3cyclictest87714-21kworker/u8:3-ext4-rsv-conversion20:00:002
65924992016,3cyclictest0-21swapper/222:50:002
65924992016,3cyclictest0-21swapper/222:44:132
65924991916,2cyclictest250910-21cron22:15:002
65924991915,3cyclictest0-21swapper/219:45:002
65924991915,2cyclictest0-21swapper/219:18:592
65924991914,4cyclictest2874338-21kworker/u8:0+flush-8:022:24:002
1810852190,2chrt481ktimers/321:04:093
65924991814,3cyclictest122408-21cron20:05:002
65924991814,2cyclictest0-21swapper/223:39:042
65924991814,2cyclictest0-21swapper/221:35:002
65924991814,2cyclictest0-21swapper/221:24:012
65924991814,2cyclictest0-21swapper/219:25:012
65924991813,3cyclictest0-21swapper/223:45:002
65924991810,6cyclictest235257-21tr21:59:032
6592499172,11cyclictest380175-21/usr/sbin/munin00:29:152
6592499171,6cyclictest167672-21tune2fs20:53:592
65924991713,2cyclictest0-21swapper/221:55:002
65924991713,2cyclictest0-21swapper/221:18:542
65924991712,3cyclictest221301-21cron21:45:002
6592499170,8cyclictest186640-21grep21:09:132
6592499170,8cyclictest186640-21grep21:09:122
6592499162,13cyclictest0-21swapper/222:54:132
6592499162,10cyclictest309817-21ps23:14:112
65924991613,2cyclictest0-21swapper/223:49:312
65924991613,2cyclictest0-21swapper/220:33:082
65924991610,4cyclictest0-21swapper/219:54:022
2514752152,8sleep30-21swapper/322:18:533
3944212132,7sleep30-21swapper/300:40:013
3145112132,7sleep30-21swapper/323:19:083
2995772132,7sleep30-21swapper/323:04:073
2805692132,6sleep30-21swapper/322:45:013
2610492133,6sleep30-21swapper/322:25:113
2015572132,7sleep30-21swapper/321:25:003
1119692132,7sleep30-21swapper/319:54:083
1020282132,7sleep30-21swapper/319:44:093
874702122,5sleep30-21swapper/319:30:013
735602122,6sleep30-21swapper/319:18:573
660272122,6sleep30-21swapper/319:10:113
3855482123,6sleep30-21swapper/300:34:003
3698072122,6sleep30-21swapper/300:15:013
3310042122,6sleep30-21swapper/323:38:583
3300692122,6sleep30-21swapper/323:35:013
3205742122,6sleep30-21swapper/323:27:103
3053812122,5sleep30-21swapper/323:10:013
2410912122,5sleep30-21swapper/322:05:013
2163972122,6sleep30-21swapper/321:40:013
1918802122,5sleep30-21swapper/321:15:013
1911312122,6sleep30-21swapper/321:14:093
1867512122,6sleep30-21swapper/321:10:003
1867512122,6sleep30-21swapper/321:10:003
1747342122,6sleep30-21swapper/320:59:043
1662772122,6sleep30-21swapper/320:49:093
1518082122,6sleep30-21swapper/320:34:133
1428562122,6sleep30-21swapper/320:28:553
1326412122,6sleep30-21swapper/320:17:113
1281502122,6sleep30-21swapper/320:13:583
1068292122,6sleep30-21swapper/319:49:033
945242112,5sleep30-21swapper/319:39:043
780832112,5sleep30-21swapper/319:22:553
3800652112,5sleep30-21swapper/300:27:293
3749512112,5sleep30-21swapper/300:20:113
3652752112,6sleep30-21swapper/300:12:413
3606832112,5sleep30-21swapper/300:08:593
3551122112,6sleep30-21swapper/300:02:133
3500082112,5sleep30-21swapper/323:55:113
3492302112,5sleep30-21swapper/323:54:113
3404272112,6sleep30-21swapper/323:48:173
3352912112,6sleep30-21swapper/323:40:433
3156082113,5sleep30-21swapper/323:22:053
3010122112,5sleep30-21swapper/323:08:573
2907202113,5sleep30-21swapper/322:56:413
2857762112,6sleep30-21swapper/322:51:363
2760272112,5sleep30-21swapper/322:42:123
2709472112,5sleep30-21swapper/322:35:463
2660682112,5sleep30-21swapper/322:31:333
2563462112,5sleep30-21swapper/322:23:113
2462732112,5sleep30-21swapper/322:11:113
2315282112,5sleep30-21swapper/321:57:103
2265792113,5sleep30-21swapper/321:51:453
2215952112,5sleep30-21swapper/321:45:473
2157012112,5sleep30-21swapper/321:39:063
2071112112,5sleep30-21swapper/321:33:553
2018562112,5sleep30-21swapper/321:25:493
1676112112,5sleep30-21swapper/320:53:593
1572812112,6sleep30-21swapper/320:41:113
1564532112,5sleep30-21swapper/320:39:113
1178502112,5sleep30-21swapper/320:01:143
1128052112,6sleep30-21swapper/319:55:053
877112102,5sleep30-21swapper/319:30:053
1228232102,5sleep30-21swapper/320:07:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional