You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 06:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Jul 28, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2834911130,59ptp4l0-21swapper/119:07:451
94080890,84rtkit-daemon0-21swapper/419:06:116
1380026856,9sleep110-21swapper/1119:05:283
1479326755,10sleep00-21swapper/019:08:290
1468026654,10sleep70-21swapper/719:06:489
1370326654,9sleep100-21swapper/1019:05:272
1160826654,10sleep60-21swapper/619:05:118
1469226352,9sleep50-21swapper/519:06:587
263492530,0sleep50-21swapper/500:15:177
319922500,0sleep90-21swapper/923:00:1311
153472480,0sleep70-21swapper/723:20:139
253832190,0sleep00-21swapper/020:45:140
283491170,0ptp4l0-21swapper/123:10:231
1529499173,9cyclictest0-21swapper/1121:30:013
1524999173,9cyclictest0-21swapper/121:05:001
283491150,1ptp4l21-21ksoftirqd/122:15:251
283491150,0ptp4l0-21swapper/122:25:231
283491150,0ptp4l0-21swapper/121:05:171
283491150,0ptp4l0-21swapper/100:20:131
277842150,0sleep100-21swapper/1022:55:132
1529499152,8cyclictest0-21swapper/1120:25:003
1529499152,2cyclictest0-21swapper/1121:40:163
1529499152,2cyclictest0-21swapper/1121:40:163
1528599153,8cyclictest0-21swapper/923:55:0111
1528599153,7cyclictest0-21swapper/923:15:0111
1528599153,7cyclictest0-21swapper/919:40:0011
1525899153,7cyclictest0-21swapper/321:15:005
283491140,1ptp4l21-21ksoftirqd/121:41:221
283491140,1ptp4l21-21ksoftirqd/121:41:221
283491140,1ptp4l21-21ksoftirqd/120:56:421
283491140,0ptp4l0-21swapper/123:05:151
283491140,0ptp4l0-21swapper/120:05:151
283491140,0ptp4l0-21swapper/119:40:191
1529499142,7cyclictest0-21swapper/1122:50:013
1529499142,7cyclictest0-21swapper/1100:15:013
1526799142,8cyclictest0-21swapper/523:50:017
15267991411,2cyclictest0-21swapper/519:15:477
323222130,0sleep100-21swapper/1021:35:172
323222130,0sleep100-21swapper/1021:35:172
283491130,1ptp4l21-21ksoftirqd/122:06:001
283491130,1ptp4l21-21ksoftirqd/100:15:221
283491130,0ptp4l0-21swapper/121:20:231
283491130,0ptp4l0-21swapper/120:11:511
1529499132,7cyclictest0-21swapper/1122:15:013
1528599133,6cyclictest0-21swapper/900:20:0111
1528599132,6cyclictest0-21swapper/922:40:0111
1527699132,10cyclictest15969-21kworker/u24:021:20:009
1526799132,7cyclictest0-21swapper/522:25:017
1526799132,6cyclictest0-21swapper/522:10:017
1526799132,6cyclictest0-21swapper/500:40:017
1525899131,7cyclictest0-21swapper/300:15:015
1524999132,7cyclictest0-21swapper/123:30:011
1524999132,7cyclictest0-21swapper/100:25:011
1524599134,6cyclictest0-21swapper/020:05:000
115982136,5sleep90-21swapper/919:05:0211
110792132,8sleep80-21swapper/819:05:0110
283491120,5ptp4l21-21ksoftirqd/119:35:251
283491120,3ptp4l6832-21expr21:45:111
283491120,3ptp4l6832-21expr21:45:111
283491120,3ptp4l24714-21latency_hist20:45:011
283491120,1ptp4l21-21ksoftirqd/121:39:221
283491120,1ptp4l21-21ksoftirqd/121:39:221
283491120,1ptp4l21-21ksoftirqd/121:25:241
283491120,1ptp4l21-21ksoftirqd/120:20:161
283491120,0ptp4l0-21swapper/123:45:011
283491120,0ptp4l0-21swapper/123:35:191
283491120,0ptp4l0-21swapper/121:18:421
283491120,0ptp4l0-21swapper/121:10:151
283491120,0ptp4l0-21swapper/119:30:191
160062120,0sleep00-21swapper/021:55:210
1529499122,6cyclictest0-21swapper/1119:25:003
1529099120,2cyclictest20162-21munin-run00:10:002
1528599122,7cyclictest0-21swapper/921:30:0111
1528599122,7cyclictest0-21swapper/921:30:0111
1527699123,7cyclictest0-21swapper/720:55:019
1527699122,7cyclictest0-21swapper/721:25:269
1527699120,11cyclictest2147-21/usr/sbin/munin23:45:139
1526799122,6cyclictest0-21swapper/523:30:017
1526799122,6cyclictest0-21swapper/522:00:007
1525899122,6cyclictest0-21swapper/321:40:015
1525899122,6cyclictest0-21swapper/321:40:015
1524999122,6cyclictest0-21swapper/123:55:261
1524999122,6cyclictest0-21swapper/119:14:591
1524599122,6cyclictest0-21swapper/023:40:010
15245991210,1cyclictest31988-21turbostat20:59:550
1524599120,11cyclictest28924-21grep22:55:170
299362110,0sleep00-21swapper/019:25:180
283491110,3ptp4l20381-21cpuspeed_turbos22:45:141
283491110,3ptp4l20008-21/usr/sbin/munin20:35:251
283491110,2ptp4l2950-21gltestperf20:15:151
283491110,2ptp4l23141-21fschecks_count23:30:151
283491110,2ptp4l17396-21fw_forwarded_lo22:40:151
283491110,2ptp4l1726-21/usr/sbin/munin22:20:131
283491110,2ptp4l16996-21awk21:55:271
283491110,2ptp4l14223-21df20:30:131
283491110,1ptp4l21-21ksoftirqd/120:00:221
283491110,1ptp4l18711-21date19:55:001
283491110,1ptp4l18649-21df_inode22:00:131
283491110,0ptp4l0-21swapper/122:30:011
283491110,0ptp4l0-21swapper/121:50:241
283491110,0ptp4l0-21swapper/121:50:241
283491110,0ptp4l0-21swapper/119:20:121
283491110,0ptp4l0-21swapper/100:10:251
1529499111,6cyclictest0-21swapper/1123:00:013
1529499110,6cyclictest0-21swapper/1123:34:553
1528599110,6cyclictest0-21swapper/923:10:0011
1527699119,1cyclictest0-21swapper/700:25:209
1527699114,6cyclictest0-21swapper/723:40:019
1527699112,8cyclictest0-21swapper/722:35:019
1527699112,8cyclictest0-21swapper/720:00:009
1527699112,8cyclictest0-21swapper/719:50:009
1527699112,7cyclictest0-21swapper/723:45:019
1527699112,7cyclictest0-21swapper/719:10:259
1526399110,10cyclictest5212-21irqrtprio19:35:186
1526399110,10cyclictest13278-40dnf21:51:516
1526399110,10cyclictest13278-40dnf21:51:516
1525899112,6cyclictest0-21swapper/323:20:245
1525899111,6cyclictest0-21swapper/319:45:005
1525499110,2cyclictest31347-21tune2fs20:10:164
1524999118,2cyclictest8672-21turbostat23:55:001
1524599113,5cyclictest0-21swapper/023:30:250
1524599113,5cyclictest0-21swapper/000:00:240
1524599112,6cyclictest0-21swapper/023:05:220
1524599112,6cyclictest0-21swapper/021:15:010
1524599110,9cyclictest0-21swapper/023:18:040
283491100,4ptp4l26030-21/usr/sbin/munin20:45:171
283491100,1ptp4l8725-21perl00:35:151
283491100,1ptp4l26819-21users22:50:251
283491100,1ptp4l21-21ksoftirqd/123:15:281
283491100,1ptp4l21-21ksoftirqd/121:30:171
283491100,1ptp4l21-21ksoftirqd/121:30:171
283491100,1ptp4l19399-21sed19:15:001
283491100,1ptp4l15793-21fw_forwarded_lo23:20:151
283491100,1ptp4l14263-21smartctl00:00:151
283491100,0ptp4l0-21swapper/123:00:131
283491100,0ptp4l0-21swapper/120:50:161
283491100,0ptp4l0-21swapper/100:30:191
1529499100,9cyclictest25330-21ps22:50:173
1529499100,4cyclictest0-21swapper/1123:39:553
1529499100,1cyclictest0-21swapper/1119:44:553
1528599108,1cyclictest0-21swapper/921:25:1511
1528599100,2cyclictest0-21swapper/921:55:1611
1528199100,1cyclictest0-21swapper/823:00:2010
1527699103,6cyclictest0-21swapper/719:25:009
1527699101,8cyclictest0-21swapper/700:00:019
1527699100,9cyclictest0-21swapper/700:35:239
1527699100,1cyclictest0-21swapper/721:50:199
1527699100,1cyclictest0-21swapper/721:50:199
1527699100,1cyclictest0-21swapper/720:15:199
1527299100,9cyclictest1274-21irqrtprio00:25:178
1527299100,3cyclictest27216-21awk22:55:008
1527299100,1cyclictest0-21swapper/621:45:008
1527299100,1cyclictest0-21swapper/621:45:008
1526799102,5cyclictest0-21swapper/519:45:007
1526799101,6cyclictest0-21swapper/521:25:007
1526399100,9cyclictest29147-21cpuspeed_turbos20:50:136
1525899107,2cyclictest17592-21needreboot19:10:205
1524999102,5cyclictest0-21swapper/120:00:001
1524599109,1cyclictest0-21swapper/022:45:130
1524599102,6cyclictest0-21swapper/000:30:010
1524599102,5cyclictest0-21swapper/023:55:010
1524599102,5cyclictest0-21swapper/023:50:010
1524599102,5cyclictest0-21swapper/023:05:010
1524599102,5cyclictest0-21swapper/022:10:010
1524599102,4cyclictest0-21swapper/023:45:010
9748290,0sleep60-21swapper/600:35:208
9408090,2rtkit-daemon0-21swapper/919:53:1711
9408090,0rtkit-daemon0-21swapper/923:25:1211
28359190,1phc2sys0-21swapper/219:05:234
28349190,1ptp4l21-21ksoftirqd/123:40:121
28349190,1ptp4l21-21ksoftirqd/122:55:141
28349190,1ptp4l21-21ksoftirqd/122:35:121
28349190,0ptp4l0-21swapper/122:10:111
28349190,0ptp4l0-21swapper/120:25:131
28349190,0ptp4l0-21swapper/119:45:151
28349190,0ptp4l0-21swapper/119:25:011
28349190,0ptp4l0-21swapper/100:05:131
152859998,1cyclictest0-21swapper/922:55:1411
152859997,1cyclictest0-21swapper/919:15:4711
152859997,1cyclictest0-21swapper/900:35:0111
152859990,8cyclictest0-21swapper/920:50:0011
152859990,1cyclictest0-21swapper/919:25:0011
152819998,1cyclictest0-21swapper/820:10:2310
152819990,8cyclictest0-21swapper/820:35:0110
152819990,1cyclictest0-21swapper/820:35:1510
152769998,1cyclictest0-21swapper/722:35:219
152769998,1cyclictest0-21swapper/719:30:129
152769990,9cyclictest0-21swapper/723:30:159
152769990,9cyclictest0-21swapper/723:10:149
152769990,9cyclictest0-21swapper/722:50:019
152769990,9cyclictest0-21swapper/720:40:239
152769990,9cyclictest0-21swapper/720:40:009
152769990,3cyclictest0-21swapper/723:50:169
152769990,1cyclictest0-21swapper/722:15:189
152769990,1cyclictest0-21swapper/722:00:139
152769990,1cyclictest0-21swapper/700:10:249
152729992,6cyclictest0-21swapper/623:25:018
152729992,6cyclictest0-21swapper/621:15:018
152729992,6cyclictest0-21swapper/619:40:008
152729992,3cyclictest0-21swapper/620:25:008
152729991,7cyclictest30139-21chrt21:31:378
152729991,7cyclictest30139-21chrt21:31:378
152729990,9cyclictest0-21swapper/621:00:168
152729990,8cyclictest31031-21sed00:20:238
152729990,8cyclictest0-21swapper/621:50:158
152729990,8cyclictest0-21swapper/621:50:158
152729990,8cyclictest0-21swapper/619:50:008
152729990,8cyclictest0-21swapper/619:25:008
152729990,1cyclictest0-21swapper/623:15:158
152679990,8cyclictest6135-21users23:05:247
152679990,2cyclictest15889-21hddtemp_smartct23:20:167
152639998,1cyclictest0-21swapper/400:35:156
152639990,9cyclictest0-21swapper/400:33:046
152589998,1cyclictest0-21swapper/320:45:005
152589997,2cyclictest1007-21snmpd21:46:515
152589997,2cyclictest1007-21snmpd21:46:515
152589991,5cyclictest0-21swapper/323:35:015
152589991,4cyclictest0-21swapper/300:15:015
152549998,1cyclictest15239-21sed21:15:004
152549998,1cyclictest0-21swapper/221:35:244
152549998,1cyclictest0-21swapper/221:35:244
152549998,1cyclictest0-21swapper/220:15:134
152459991,5cyclictest0-21swapper/021:40:000
152459991,5cyclictest0-21swapper/021:40:000
152459991,5cyclictest0-21swapper/000:15:210
152459990,9cyclictest0-21swapper/000:07:070
12137291,6sleep30-21swapper/319:05:205
152949984,3cyclictest1515-21date20:15:003
152949980,1cyclictest0-21swapper/1123:40:173
152909980,7cyclictest1009-21runrttasks20:25:322
152909980,1cyclictest0-21swapper/1022:35:122
152909980,1cyclictest0-21swapper/1022:25:212
152909980,1cyclictest0-21swapper/1019:50:362
152859987,1cyclictest0-21swapper/923:20:2511
152859987,1cyclictest0-21swapper/921:05:1511
152859985,2cyclictest17782-21irqrtprio22:40:1811
152859982,4cyclictest0-21swapper/921:10:1411
152859981,6cyclictest0-21swapper/922:50:0111
152859980,8cyclictest0-21swapper/920:20:1311
152859980,8cyclictest0-21swapper/920:15:1611
152859980,1cyclictest0-21swapper/921:40:1311
152859980,1cyclictest0-21swapper/921:40:1311
152859980,1cyclictest0-21swapper/920:05:0011
152859980,1cyclictest0-21swapper/919:10:2411
152819987,1cyclictest0-21swapper/820:51:4110
152819987,1cyclictest0-21swapper/819:35:2510
152819987,1cyclictest0-21swapper/800:25:2510
152819980,8cyclictest0-21swapper/823:10:2610
152819980,8cyclictest0-21swapper/822:20:1810
152819980,8cyclictest0-21swapper/800:10:2010
152819980,8cyclictest0-21swapper/800:05:1710
152819980,7cyclictest0-21swapper/820:26:1410
152819980,1cyclictest0-21swapper/821:54:3610
152819980,1cyclictest0-21swapper/821:54:3610
152819980,1cyclictest0-21swapper/821:15:2110
152819980,1cyclictest0-21swapper/819:15:4510
152819980,1cyclictest0-21swapper/819:10:2110
152769987,1cyclictest0-21swapper/722:05:209
152769987,1cyclictest0-21swapper/720:25:129
152769987,1cyclictest0-21swapper/720:20:139
152769987,1cyclictest0-21swapper/720:10:229
152769986,1cyclictest0-21swapper/720:05:249
152769985,2cyclictest9683-21munin-run21:50:009
152769985,2cyclictest9683-21munin-run21:50:009
152769980,1cyclictest0-21swapper/720:00:259
152729987,1cyclictest0-21swapper/600:10:258
152729983,4cyclictest0-21swapper/622:35:018
152729982,5cyclictest115650irq/18-p34p123:55:428
152729982,5cyclictest115650irq/18-p34p123:50:308
152729982,5cyclictest115650irq/18-p34p123:30:118
152729982,5cyclictest115650irq/18-p34p122:05:078
152729982,5cyclictest115650irq/18-p34p121:47:248
152729982,5cyclictest115650irq/18-p34p121:47:248
152729982,5cyclictest115650irq/18-p34p100:07:208
152729982,5cyclictest0-21swapper/621:40:018
152729982,5cyclictest0-21swapper/621:40:018
152729982,5cyclictest0-21swapper/620:45:018
152729981,6cyclictest51-21ksoftirqd/623:36:078
152729981,6cyclictest51-21ksoftirqd/621:11:408
152729981,6cyclictest0-21swapper/623:10:258
152729980,8cyclictest0-21swapper/619:30:138
152729980,8cyclictest0-21swapper/619:25:148
152729980,7cyclictest0-21swapper/623:00:128
152729980,7cyclictest0-21swapper/622:25:018
152729980,7cyclictest0-21swapper/622:00:148
152729980,7cyclictest0-21swapper/622:00:008
152729980,7cyclictest0-21swapper/619:55:218
152729980,7cyclictest0-21swapper/619:50:218
152729980,2cyclictest4895-21awk23:50:018
152729980,2cyclictest31646-21/usr/sbin/munin20:50:258
152729980,2cyclictest0-21swapper/622:35:128
152729980,1cyclictest0-21swapper/623:25:148
152729980,1cyclictest0-21swapper/622:55:278
152729980,1cyclictest0-21swapper/620:35:198
152729980,1cyclictest0-21swapper/620:10:148
152729980,1cyclictest0-21swapper/619:15:028
152679984,3cyclictest20147-21grep20:35:257
152639987,1cyclictest0-21swapper/423:40:126
152639987,1cyclictest0-21swapper/421:15:206
152639987,1cyclictest0-21swapper/420:05:006
152639980,8cyclictest0-21swapper/419:45:236
152639980,7cyclictest31395-21latency_hist19:29:596
152639980,3cyclictest9008-21cat23:10:206
152589987,1cyclictest0-21swapper/319:20:245
152589987,1cyclictest0-21swapper/300:35:215
152589982,4cyclictest0-21swapper/320:40:005
152589981,5cyclictest0-21swapper/300:30:015
152589980,1cyclictest0-21swapper/323:35:195
152589980,1cyclictest0-21swapper/300:05:235
152549987,1cyclictest0-21swapper/223:20:244
152549987,1cyclictest0-21swapper/223:10:014
152549987,1cyclictest0-21swapper/223:00:174
152549987,1cyclictest0-21swapper/222:05:154
152549987,1cyclictest0-21swapper/222:05:014
152549987,1cyclictest0-21swapper/221:55:164
152549987,1cyclictest0-21swapper/221:50:134
152549987,1cyclictest0-21swapper/221:50:134
152549987,1cyclictest0-21swapper/221:40:174
152549987,1cyclictest0-21swapper/221:40:174
152549987,1cyclictest0-21swapper/221:30:244
152549987,1cyclictest0-21swapper/221:30:244
152549987,1cyclictest0-21swapper/220:50:204
152549987,1cyclictest0-21swapper/219:50:214
152549987,1cyclictest0-21swapper/200:10:224
152549986,1cyclictest5132-21expr22:25:134
152549980,8cyclictest0-21swapper/223:50:244
152549980,7cyclictest0-21swapper/222:50:314
152549980,7cyclictest0-21swapper/220:38:014
152549980,1cyclictest0-21swapper/222:20:114
152549980,1cyclictest0-21swapper/220:55:234
152459987,1cyclictest0-21swapper/019:45:240
152459985,2cyclictest10720-21sed21:05:260
152459982,3cyclictest0-21swapper/022:35:010
152459981,4cyclictest0-21swapper/021:50:010
152459981,4cyclictest0-21swapper/021:50:010
152459981,4cyclictest0-21swapper/020:10:000
152459981,4cyclictest0-21swapper/000:25:010
152459980,6cyclictest0-21swapper/021:41:330
152459980,6cyclictest0-21swapper/021:41:330
152459980,1cyclictest0-21swapper/019:30:160
152459980,1cyclictest0-21swapper/000:40:010
10719280,0sleep100-21swapper/1021:05:262
28359170,6phc2sys0-21swapper/223:25:204
2100270,0sleep50-21swapper/520:55:207
152949976,1cyclictest0-21swapper/1123:17:513
152949976,1cyclictest0-21swapper/1123:11:063
152949976,1cyclictest0-21swapper/1123:06:083
152949976,1cyclictest0-21swapper/1122:35:593
152949976,1cyclictest0-21swapper/1122:26:073
152949976,1cyclictest0-21swapper/1122:19:203
152949976,1cyclictest0-21swapper/1121:15:133
152949976,1cyclictest0-21swapper/1121:13:063
152949976,1cyclictest0-21swapper/1120:52:083
152949976,1cyclictest0-21swapper/1120:45:073
152949976,1cyclictest0-21swapper/1120:00:143
152949976,1cyclictest0-21swapper/1119:35:173
152949976,1cyclictest0-21swapper/1119:15:493
152949976,1cyclictest0-21swapper/1119:10:203
152949976,1cyclictest0-21swapper/1100:38:293
152949976,1cyclictest0-21swapper/1100:22:193
152949976,1cyclictest0-21swapper/1100:15:203
152949976,1cyclictest0-21swapper/1100:03:163
152949974,2cyclictest1009-21runrttasks23:55:393
152949974,2cyclictest1009-21runrttasks00:26:073
152949970,7cyclictest0-21swapper/1123:50:133
152949970,7cyclictest0-21swapper/1123:25:553
152949970,7cyclictest0-21swapper/1123:20:113
152949970,7cyclictest0-21swapper/1121:45:063
152949970,7cyclictest0-21swapper/1121:45:063
152949970,7cyclictest0-21swapper/1121:00:243
152949970,7cyclictest0-21swapper/1120:30:193
152949970,7cyclictest0-21swapper/1120:26:523
152949970,1cyclictest0-21swapper/1123:49:193
152949970,1cyclictest0-21swapper/1123:00:093
152949970,1cyclictest0-21swapper/1122:42:493
152949970,1cyclictest0-21swapper/1122:30:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional