You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-21 - 04:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Jul 21, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
283491990,89ptp4l0-21swapper/119:05:231
94080800,75rtkit-daemon0-21swapper/1119:09:593
469527759,9sleep100-21swapper/1019:07:592
447027060,8sleep50-21swapper/519:05:317
447126950,10sleep60-21swapper/619:05:318
470226755,10sleep40-21swapper/419:08:066
472626655,9sleep00-21swapper/019:08:250
451926553,10sleep70-21swapper/719:06:089
247522490,0sleep90-21swapper/921:40:1411
527999250,1cyclictest0-21swapper/1020:25:222
524799201,14cyclictest0-21swapper/420:20:006
5279991918,1cyclictest0-21swapper/1023:20:222
526999180,1cyclictest0-21swapper/823:49:5510
523799182,10cyclictest0-21swapper/221:50:004
526499173,8cyclictest0-21swapper/720:50:009
524299173,9cyclictest0-21swapper/319:35:015
522699173,9cyclictest0-21swapper/022:45:010
522699173,8cyclictest0-21swapper/022:20:250
522699173,8cyclictest0-21swapper/022:20:240
283491170,0ptp4l0-21swapper/119:55:181
528399162,8cyclictest0-21swapper/1122:05:013
527399161,3cyclictest0-21swapper/923:15:1711
526999164,7cyclictest0-21swapper/819:40:0010
526999163,8cyclictest0-21swapper/823:10:0110
526499164,7cyclictest0-21swapper/723:45:009
525199162,8cyclictest0-21swapper/523:45:017
524299162,8cyclictest0-21swapper/322:30:015
524299162,8cyclictest0-21swapper/322:30:005
524299162,8cyclictest0-21swapper/320:50:265
523799160,11cyclictest0-21swapper/222:05:014
523199163,8cyclictest0-21swapper/120:45:001
523199162,8cyclictest0-21swapper/122:55:001
523199161,9cyclictest15200-21kworker/u24:123:40:001
522699162,8cyclictest0-21swapper/023:50:010
522699162,8cyclictest0-21swapper/021:25:010
47602167,7sleep80-21swapper/819:08:5810
283491160,0ptp4l0-21swapper/120:15:171
527399154,7cyclictest0-21swapper/920:45:0011
527399152,8cyclictest0-21swapper/920:25:0011
526499153,7cyclictest0-21swapper/719:25:019
526499152,8cyclictest0-21swapper/723:35:019
526499152,8cyclictest0-21swapper/721:35:009
526499152,8cyclictest0-21swapper/719:15:259
526499152,8cyclictest0-21swapper/700:10:009
525199153,7cyclictest0-21swapper/523:15:017
525199153,7cyclictest0-21swapper/521:35:017
5242991512,3cyclictest0-21swapper/322:35:165
523799153,8cyclictest0-21swapper/222:45:004
523799152,8cyclictest0-21swapper/220:00:004
523199153,6cyclictest0-21swapper/122:10:011
523199153,6cyclictest0-21swapper/122:10:001
523199152,8cyclictest0-21swapper/100:30:001
523199152,7cyclictest0-21swapper/121:50:011
522699153,8cyclictest0-21swapper/021:10:000
522699153,7cyclictest0-21swapper/021:35:000
283491150,1ptp4l21-21ksoftirqd/123:13:151
283491150,1ptp4l21-21ksoftirqd/120:35:201
264591150,0getstats0-21swapper/122:30:161
74912140,0sleep40-21swapper/400:05:126
528399143,7cyclictest0-21swapper/1120:55:013
528399142,8cyclictest0-21swapper/1121:15:003
528399142,7cyclictest0-21swapper/1120:20:003
528399141,8cyclictest0-21swapper/1123:00:013
527999146,8cyclictest0-21swapper/1020:30:212
527399143,7cyclictest0-21swapper/920:30:0011
527399142,7cyclictest0-21swapper/920:00:0011
527399142,7cyclictest0-21swapper/919:30:2411
526999143,7cyclictest0-21swapper/823:45:0010
526999142,7cyclictest0-21swapper/820:40:0110
526999142,7cyclictest0-21swapper/800:25:0110
526499143,7cyclictest0-21swapper/700:05:009
526499142,7cyclictest0-21swapper/721:17:109
525699141,9cyclictest27757-21kworker/u24:121:45:008
525199143,7cyclictest0-21swapper/520:30:007
525199142,7cyclictest0-21swapper/523:15:257
525199142,7cyclictest0-21swapper/519:20:007
524799141,2cyclictest0-21swapper/420:05:176
523799143,8cyclictest0-21swapper/223:15:004
523199143,7cyclictest0-21swapper/121:05:011
523199142,8cyclictest549-21kworker/u24:219:25:001
523199142,7cyclictest0-21swapper/121:55:001
522699142,7cyclictest0-21swapper/000:20:260
522699142,7cyclictest0-21swapper/000:15:010
283491140,1ptp4l21-21ksoftirqd/123:42:191
283491140,0ptp4l0-21swapper/119:10:511
528399132,7cyclictest0-21swapper/1121:30:003
5283991311,1cyclictest0-21swapper/1120:34:563
527999132,7cyclictest0-21swapper/1021:45:012
527999130,12cyclictest0-21swapper/1019:24:552
527399132,7cyclictest0-21swapper/920:20:0011
526499132,7cyclictest0-21swapper/722:15:019
526499132,7cyclictest0-21swapper/722:15:019
525699132,7cyclictest0-21swapper/623:50:008
525699132,7cyclictest0-21swapper/622:30:018
525699132,7cyclictest0-21swapper/622:30:008
525699132,7cyclictest0-21swapper/621:25:258
525199132,7cyclictest0-21swapper/519:25:007
524799132,6cyclictest0-21swapper/420:00:236
524299133,6cyclictest0-21swapper/322:20:015
524299133,6cyclictest0-21swapper/322:20:015
524299132,8cyclictest0-21swapper/300:15:005
524299132,6cyclictest0-21swapper/320:40:005
523799132,7cyclictest0-21swapper/222:45:004
523799132,7cyclictest0-21swapper/221:10:014
523799132,7cyclictest0-21swapper/219:15:014
523199132,7cyclictest0-21swapper/121:10:001
522699133,6cyclictest0-21swapper/022:20:000
522699133,6cyclictest0-21swapper/022:20:000
522699132,7cyclictest0-21swapper/023:05:000
522699132,7cyclictest0-21swapper/021:50:010
522699132,6cyclictest0-21swapper/000:35:010
522699131,8cyclictest0-21swapper/023:10:010
5226991310,2cyclictest8752-21turbostat19:15:000
283491130,1ptp4l21-21ksoftirqd/122:51:501
283491130,1ptp4l21-21ksoftirqd/122:40:381
283491130,0ptp4l0-21swapper/123:55:251
283491130,0ptp4l0-21swapper/100:22:351
269792130,0sleep10-21swapper/119:35:161
528399122,6cyclictest0-21swapper/1100:05:003
527399121,7cyclictest0-21swapper/923:25:0011
526999123,6cyclictest0-21swapper/819:20:2510
526999121,6cyclictest0-21swapper/800:05:2210
526499122,7cyclictest0-21swapper/719:25:269
525699120,11cyclictest2570-21fschecks_time21:10:148
525199123,5cyclictest0-21swapper/522:40:007
525199122,6cyclictest0-21swapper/523:05:007
524299122,7cyclictest0-21swapper/319:20:005
524299122,6cyclictest0-21swapper/323:40:225
522699123,6cyclictest0-21swapper/023:55:010
522699120,2cyclictest13656-21perl20:00:140
45492126,5sleep90-21swapper/919:06:3411
317962120,0sleep30-21swapper/320:25:125
283491120,1ptp4l21-21ksoftirqd/123:20:061
283491120,1ptp4l21-21ksoftirqd/120:45:211
283491120,1ptp4l0-21swapper/100:00:141
283491120,0ptp4l0-21swapper/119:30:531
528399112,5cyclictest0-21swapper/1122:40:013
528399112,5cyclictest0-21swapper/1120:00:003
528399111,6cyclictest0-21swapper/1121:45:013
527999113,5cyclictest0-21swapper/1000:40:012
526999112,7cyclictest0-21swapper/821:55:0110
526999111,6cyclictest0-21swapper/822:35:0010
526999110,10cyclictest0-21swapper/821:10:1410
526499112,7cyclictest0-21swapper/723:50:259
525699111,6cyclictest0-21swapper/600:10:008
525699110,10cyclictest11000-21perl22:45:168
525199110,10cyclictest20686-21switchtime00:20:257
524299112,6cyclictest0-21swapper/323:20:015
523799112,5cyclictest0-21swapper/221:10:004
522699118,2cyclictest0-21swapper/021:25:230
522699111,6cyclictest0-21swapper/020:20:000
522699110,2cyclictest24356-21memory20:55:210
283491110,3ptp4l21-21ksoftirqd/121:40:221
283491110,2ptp4l9703-21cat21:20:141
283491110,2ptp4l7790-21fschecks_time00:05:131
283491110,2ptp4l0-21swapper/119:15:271
283491110,1ptp4l8677-40dnf21:17:101
283491110,1ptp4l22905-21cpuspeed_turbos20:55:141
283491110,1ptp4l21-21ksoftirqd/123:25:281
283491110,1ptp4l21-21ksoftirqd/121:30:121
283491110,1ptp4l0-21swapper/122:45:141
283491110,0ptp4l0-21swapper/123:50:271
264591110,0getstats0-21swapper/122:05:171
527999102,5cyclictest0-21swapper/1021:20:012
527999102,4cyclictest0-21swapper/1022:50:002
527999100,1cyclictest0-21swapper/1020:55:242
527999100,1cyclictest0-21swapper/1020:45:142
527399100,9cyclictest17106-21cut22:10:2011
527399100,9cyclictest17106-21cut22:10:1911
526999102,7cyclictest549-21kworker/u24:219:45:0010
526999102,7cyclictest0-21swapper/820:55:0010
526999100,9cyclictest0-21swapper/823:15:0110
526499100,9cyclictest0-21swapper/720:10:199
526499100,1cyclictest0-21swapper/723:55:189
525699106,3cyclictest10798-21logger22:01:008
524799100,9cyclictest0-21swapper/423:50:136
523799102,5cyclictest0-21swapper/220:25:004
523799100,9cyclictest0-21swapper/221:15:154
523799100,9cyclictest0-21swapper/220:35:004
523799100,9cyclictest0-21swapper/219:20:194
523199106,3cyclictest9097-21/usr/sbin/munin22:00:161
522699100,9cyclictest0-21swapper/000:00:110
283491100,2ptp4l14421-21gltestperf21:25:161
283491100,1ptp4l5009-21irqrtprio20:30:171
283491100,1ptp4l29028-21fw_conntrack20:20:141
283491100,1ptp4l21-21ksoftirqd/123:15:181
283491100,1ptp4l21-21ksoftirqd/121:35:151
283491100,1ptp4l21-21ksoftirqd/120:10:141
283491100,1ptp4l21-21ksoftirqd/119:20:201
283491100,1ptp4l21-21ksoftirqd/100:17:311
283491100,1ptp4l1755-21diskstats19:45:151
283491100,1ptp4l1238-21load20:25:181
283491100,0ptp4l0-21swapper/123:45:131
283491100,0ptp4l0-21swapper/123:30:131
283491100,0ptp4l0-21swapper/122:20:191
283491100,0ptp4l0-21swapper/122:20:191
264691100,1getstats27-21ksoftirqd/220:04:454
264591100,0getstats0-21swapper/122:35:171
52839998,1cyclictest801rcuc/1100:25:003
52839998,1cyclictest0-21swapper/1123:25:263
52839997,1cyclictest801rcuc/1120:40:143
52839992,6cyclictest0-21swapper/1122:15:003
52839992,6cyclictest0-21swapper/1122:15:003
52839990,8cyclictest25151-21munin-run19:35:003
52839990,8cyclictest10513-21expr22:45:133
52839990,1cyclictest0-21swapper/1123:05:133
52839990,1cyclictest0-21swapper/1119:50:173
52799998,1cyclictest0-21swapper/1023:35:122
52799991,5cyclictest0-21swapper/1000:30:002
52799990,9cyclictest0-21swapper/1022:50:002
52799990,8cyclictest31732-21ps19:40:222
52799990,8cyclictest20900-21latency_hist23:00:012
52799990,4cyclictest0-21swapper/1020:40:002
52799990,1cyclictest0-21swapper/1021:20:142
52799990,1cyclictest0-21swapper/1020:20:142
52799990,1cyclictest0-21swapper/1019:30:182
52739998,1cyclictest69-21ksoftirqd/919:30:0011
52739998,1cyclictest0-21swapper/921:50:1911
52739997,1cyclictest0-21swapper/921:30:1511
52739994,3cyclictest681rcuc/919:10:1811
52739990,8cyclictest988-21diskmemload21:35:1811
52739990,1cyclictest0-21swapper/900:20:2611
52699997,1cyclictest0-21swapper/820:25:1110
52699995,3cyclictest31158-21runrttasks20:24:5310
52699990,9cyclictest0-21swapper/822:15:1410
52699990,9cyclictest0-21swapper/822:15:1410
52699990,8cyclictest0-21swapper/822:50:1110
52699990,8cyclictest0-21swapper/821:30:1910
52649998,1cyclictest0-21swapper/721:35:149
52649996,2cyclictest1009-21runrttasks21:28:229
52649992,6cyclictest0-21swapper/722:05:009
52649990,9cyclictest0-21swapper/720:50:259
52649990,8cyclictest0-21swapper/722:25:169
52649990,8cyclictest0-21swapper/722:25:159
52649990,8cyclictest0-21swapper/722:23:009
52649990,8cyclictest0-21swapper/722:22:599
52649990,8cyclictest0-21swapper/722:17:099
52649990,8cyclictest0-21swapper/722:17:099
52649990,8cyclictest0-21swapper/720:40:159
52649990,1cyclictest0-21swapper/721:20:139
52649990,1cyclictest0-21swapper/719:40:199
52569996,2cyclictest6978-21df22:40:148
52569992,6cyclictest0-21swapper/600:00:028
52569991,7cyclictest0-21swapper/620:55:148
52569990,9cyclictest0-21swapper/621:00:138
52569990,8cyclictest20973-21/usr/sbin/munin23:40:138
52569990,8cyclictest15354-21cstates20:45:128
52569990,8cyclictest0-21swapper/619:35:188
52569990,8cyclictest0-21swapper/600:35:008
52569990,3cyclictest26115-21/usr/sbin/munin20:15:218
52569990,2cyclictest20332-21perl22:15:148
52569990,2cyclictest20332-21perl22:15:148
52569990,1cyclictest0-21swapper/621:30:218
52569990,1cyclictest0-21swapper/620:00:238
52569990,1cyclictest0-21swapper/619:15:008
52479996,2cyclictest18174-21expr22:55:146
52479993,4cyclictest14750irq/20-ata_piix22:15:016
52479993,4cyclictest14750irq/20-ata_piix22:15:006
52479990,5cyclictest0-21swapper/400:17:296
52429998,1cyclictest0-21swapper/322:50:255
52429990,9cyclictest0-21swapper/300:00:135
52379992,5cyclictest0-21swapper/219:50:254
52379991,5cyclictest0-21swapper/220:40:004
52379990,9cyclictest0-21swapper/222:05:144
52379990,8cyclictest0-21swapper/222:20:234
52379990,8cyclictest0-21swapper/222:20:234
52379990,1cyclictest0-21swapper/220:10:254
52269998,1cyclictest0-21swapper/020:20:170
52269998,1cyclictest0-21swapper/019:40:250
52269998,1cyclictest0-21swapper/019:20:170
52269996,2cyclictest32360-21turbostat23:15:010
52269990,8cyclictest0-21swapper/023:39:570
52269990,1cyclictest0-21swapper/022:30:140
28349190,1ptp4l21-21ksoftirqd/123:05:021
28349190,1ptp4l21-21ksoftirqd/123:00:151
28349190,1ptp4l21-21ksoftirqd/122:25:011
28349190,1ptp4l21-21ksoftirqd/122:25:011
28349190,1ptp4l21-21ksoftirqd/122:15:121
28349190,1ptp4l21-21ksoftirqd/122:15:121
28349190,1ptp4l21-21ksoftirqd/121:55:131
28349190,1ptp4l21-21ksoftirqd/120:50:131
28349190,1ptp4l21-21ksoftirqd/100:35:021
28349190,1ptp4l21-21ksoftirqd/100:30:131
28349190,1ptp4l21-21ksoftirqd/100:10:141
28349190,0ptp4l0-21swapper/121:10:121
28349190,0ptp4l0-21swapper/120:05:141
28349190,0ptp4l0-21swapper/120:00:001
28349190,0ptp4l0-21swapper/119:50:001
28349190,0ptp4l0-21swapper/119:40:011
52839987,1cyclictest801rcuc/1100:35:163
52839987,1cyclictest0-21swapper/1100:05:233
52839982,5cyclictest0-21swapper/1120:20:003
52839980,8cyclictest0-21swapper/1122:15:183
52839980,8cyclictest0-21swapper/1122:15:173
52839980,7cyclictest0-21swapper/1122:25:173
52839980,7cyclictest0-21swapper/1122:25:173
52839980,1cyclictest0-21swapper/1123:55:163
52839980,1cyclictest0-21swapper/1120:55:163
52839980,1cyclictest0-21swapper/1120:35:253
52839980,1cyclictest0-21swapper/1120:10:153
52739987,1cyclictest681rcuc/900:06:0411
52739987,1cyclictest0-21swapper/923:45:1711
52739987,1cyclictest0-21swapper/923:30:2111
52739987,1cyclictest0-21swapper/922:55:2611
52739987,1cyclictest0-21swapper/921:25:2011
52739980,8cyclictest0-21swapper/923:10:1911
52739980,8cyclictest0-21swapper/922:35:1811
52739980,8cyclictest0-21swapper/922:30:1611
52739980,8cyclictest0-21swapper/922:25:1211
52739980,8cyclictest0-21swapper/922:25:1211
52739980,8cyclictest0-21swapper/921:55:2011
52699987,1cyclictest0-21swapper/823:00:2510
52699980,1cyclictest0-21swapper/821:55:2010
52699980,1cyclictest0-21swapper/820:05:1710
52649987,1cyclictest0-21swapper/723:20:129
52649987,1cyclictest0-21swapper/721:05:509
52649985,2cyclictest31697-21fschecks_time22:30:149
52649984,3cyclictest1475-21cstates19:45:149
52649982,5cyclictest0-21swapper/721:50:269
52649980,8cyclictest0-21swapper/719:58:319
52649980,8cyclictest0-21swapper/700:29:549
52649980,2cyclictest0-21swapper/719:54:309
52649980,2cyclictest0-21swapper/700:16:399
52649980,1cyclictest0-21swapper/721:10:139
52649980,1cyclictest0-21swapper/719:10:259
52649980,1cyclictest0-21swapper/700:30:169
52569987,1cyclictest0-21swapper/621:50:218
52569987,1cyclictest0-21swapper/620:20:218
52569982,5cyclictest115650irq/18-p34p123:18:398
52569982,5cyclictest115650irq/18-p34p123:10:368
52569982,5cyclictest115650irq/18-p34p121:20:008
52569982,5cyclictest115650irq/18-p34p100:20:118
52569981,6cyclictest51-21ksoftirqd/622:21:168
52569981,6cyclictest51-21ksoftirqd/622:21:168
52569980,8cyclictest0-21swapper/623:30:148
52569980,8cyclictest0-21swapper/623:25:158
52569980,8cyclictest0-21swapper/623:05:158
52569980,8cyclictest0-21swapper/619:10:128
52569980,8cyclictest0-21swapper/600:10:178
52569980,7cyclictest6102-21irqrtprio23:20:188
52569980,7cyclictest5665-21munin-run22:40:008
52569980,7cyclictest4767-21/usr/sbin/munin21:55:148
52569980,7cyclictest18051-21/usr/sbin/munin19:25:278
52569980,7cyclictest16924-21cut22:10:208
52569980,7cyclictest16924-21cut22:10:198
52569980,7cyclictest16810-21df_inode23:35:148
52569980,7cyclictest0-21swapper/622:50:148
52569980,7cyclictest0-21swapper/620:25:118
52569980,7cyclictest0-21swapper/600:35:248
52569980,3cyclictest15855-21if_p34p100:15:178
52569980,2cyclictest9397-21ps20:35:228
52569980,2cyclictest29191-21/usr/sbin/munin23:50:198
52569980,2cyclictest2105-21/usr/sbin/munin19:45:158
52569980,2cyclictest19777-21fw_conntrack20:50:158
52569980,2cyclictest11876-21fschecks_count20:40:148
52569980,1cyclictest0-21swapper/623:55:148
52569980,1cyclictest0-21swapper/623:00:198
52569980,1cyclictest0-21swapper/600:25:218
52519985,2cyclictest22527-21turbostat22:20:017
52519985,2cyclictest22527-21turbostat22:20:007
52519985,2cyclictest14303-21turbostat20:49:567
52519985,2cyclictest12095-21turbostat20:00:017
52519980,1cyclictest0-21swapper/521:55:017
52479987,1cyclictest0-21swapper/421:10:136
52479985,2cyclictest0-21swapper/422:25:146
52479985,2cyclictest0-21swapper/422:25:136
52479980,7cyclictest7965-21cat23:25:016
52479980,1cyclictest0-21swapper/422:00:016
52479980,1cyclictest0-21swapper/419:40:196
52429987,1cyclictest0-21swapper/322:30:175
52429987,1cyclictest0-21swapper/319:55:145
52429986,1cyclictest27354-21turbostat20:24:565
52429985,2cyclictest6596-21perl21:15:145
52429980,8cyclictest0-21swapper/323:50:225
52429980,8cyclictest0-21swapper/323:45:155
52429980,8cyclictest0-21swapper/321:35:215
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional