You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-31 - 02:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Sun Aug 31, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
223099750,3rtkit-daemon2229-21rtkit-daemon20:56:211
948199704,14cyclictest0-21swapper/318:47:543
947799678,53cyclictest41ktimersoftd/022:00:210
947799677,53cyclictest0-21swapper/021:40:190
947799665,54cyclictest0-21swapper/020:40:190
947799656,50cyclictest0-21swapper/022:20:080
9479996434,10cyclictest30908-21tr19:10:241
947799645,52cyclictest0-21swapper/021:10:330
94332642,54sleep10-21swapper/121:49:341
947799635,51cyclictest0-21swapper/020:20:160
947799634,52cyclictest0-21swapper/021:05:150
140922633,52sleep00-21swapper/019:41:570
947799626,49cyclictest0-21swapper/019:05:250
947799625,50cyclictest0-21swapper/019:55:230
947799625,49cyclictest0-21swapper/019:35:350
947799624,51cyclictest0-21swapper/020:45:320
947799624,51cyclictest0-21swapper/020:45:320
947799615,49cyclictest0-21swapper/020:05:180
947799615,49cyclictest0-21swapper/020:05:180
948099606,26cyclictest0-21swapper/221:25:142
948099604,48cyclictest0-21swapper/219:30:042
948099604,48cyclictest0-21swapper/219:30:042
947799605,48cyclictest0-21swapper/021:57:100
947799605,28cyclictest0-21swapper/021:30:230
947799605,28cyclictest0-21swapper/021:30:230
947799604,49cyclictest0-21swapper/019:30:320
947799603,48cyclictest0-21swapper/021:50:090
948099597,14cyclictest0-21swapper/222:07:192
948099595,15cyclictest0-21swapper/220:47:192
948099595,15cyclictest0-21swapper/220:47:192
948099594,48cyclictest0-21swapper/221:15:282
948099592,50cyclictest0-21swapper/222:33:062
947799593,24cyclictest0-21swapper/022:25:210
9481995825,26cyclictest924-21rsyslogd19:40:053
948099585,32cyclictest0-21swapper/219:10:162
948099585,27cyclictest0-21swapper/222:57:222
9480995840,10cyclictest7814-21grep22:50:322
9480995840,10cyclictest7814-21grep22:50:322
948099584,15cyclictest0-21swapper/222:42:172
9480995822,16cyclictest0-21swapper/220:15:132
9480995821,16cyclictest0-21swapper/220:50:222
9480995821,16cyclictest0-21swapper/219:30:312
9480995818,13cyclictest0-21swapper/221:35:222
9477995840,12cyclictest0-21swapper/019:20:220
947799583,9cyclictest0-21swapper/019:00:060
947799583,48cyclictest0-21swapper/022:40:180
297102583,47sleep00-21swapper/022:30:200
27162582,10sleep10-21swapper/120:25:311
265232583,47sleep00-21swapper/021:15:280
948099576,16cyclictest0-21swapper/220:36:472
948099575,32cyclictest0-21swapper/220:30:252
948099574,45cyclictest0-21swapper/218:40:152
948099574,45cyclictest0-21swapper/218:40:152
948099574,27cyclictest0-21swapper/221:45:002
948099573,12cyclictest0-21swapper/220:10:202
9480995723,15cyclictest0-21swapper/221:30:182
9480995723,15cyclictest0-21swapper/221:30:182
9480995721,12cyclictest0-21swapper/221:55:152
9480995720,13cyclictest0-21swapper/219:20:252
948099572,38cyclictest0-21swapper/219:01:232
948099572,38cyclictest0-21swapper/218:17:232
9480995719,13cyclictest0-21swapper/222:10:292
9480995719,13cyclictest0-21swapper/222:10:292
9480995718,13cyclictest0-21swapper/218:50:142
947799575,44cyclictest21692-21aten2.4-expect22:15:070
947799573,23cyclictest0-21swapper/021:20:370
9477995715,12cyclictest0-21swapper/018:41:230
9477995715,12cyclictest0-21swapper/018:41:230
223099570,6rtkit-daemon2229-21rtkit-daemon18:37:041
223099570,4rtkit-daemon2229-21rtkit-daemon19:40:391
948099565,44cyclictest0-21swapper/218:55:182
948099565,30cyclictest0-21swapper/221:20:272
948099565,30cyclictest0-21swapper/220:40:292
948099564,28cyclictest0-21swapper/217:41:172
948099564,22cyclictest0-21swapper/219:55:032
948099564,14cyclictest0-21swapper/221:00:242
9480995638,10cyclictest21960-21grep22:15:092
948099563,45cyclictest0-21swapper/220:00:152
948099563,30cyclictest0-21swapper/222:00:112
948099563,28cyclictest0-21swapper/222:20:242
948099563,26cyclictest0-21swapper/221:45:292
9480995621,15cyclictest0-21swapper/220:20:202
947999564,46cyclictest0-21swapper/120:30:271
9479995638,12cyclictest0-21swapper/121:35:191
9479995638,12cyclictest0-21swapper/120:36:571
947799563,48cyclictest0-21swapper/020:35:320
947799563,47cyclictest0-21swapper/023:00:050
947799563,46cyclictest0-21swapper/023:07:510
9477995615,12cyclictest0-21swapper/022:45:230
9481995527,8cyclictest18009-21cat21:00:013
948099554,45cyclictest0-21swapper/219:45:292
948099554,31cyclictest0-21swapper/223:05:222
948099554,13cyclictest0-21swapper/222:25:262
948099553,46cyclictest0-21swapper/219:05:232
948099553,28cyclictest0-21swapper/223:00:132
948099553,27cyclictest0-21swapper/222:45:182
948099553,24cyclictest0-21swapper/221:10:122
947799553,46cyclictest0-21swapper/020:25:140
947799553,46cyclictest0-21swapper/017:40:080
947799553,45cyclictest0-21swapper/020:55:090
947799553,25cyclictest0-21swapper/022:59:560
947799553,24cyclictest0-21swapper/021:37:530
313492553,45sleep00-21swapper/021:25:270
223099550,5rtkit-daemon2229-21rtkit-daemon20:10:231
948099546,29cyclictest0-21swapper/220:55:202
948099544,26cyclictest0-21swapper/218:45:142
948099544,13cyclictest0-21swapper/222:37:562
948099543,25cyclictest0-21swapper/220:05:152
948099543,25cyclictest0-21swapper/220:05:152
9480995418,12cyclictest0-21swapper/219:35:282
9480995415,12cyclictest0-21swapper/220:25:132
9480995412,14cyclictest0-21swapper/218:11:332
9479995437,12cyclictest0-21swapper/119:45:221
947799544,39cyclictest0-21swapper/018:26:360
947799543,46cyclictest0-21swapper/017:50:410
947799543,45cyclictest0-21swapper/022:05:080
947799543,45cyclictest0-21swapper/020:00:200
947799543,25cyclictest0-21swapper/020:50:190
947799542,35cyclictest0-21swapper/020:20:030
9477995414,12cyclictest0-21swapper/020:10:360
258442542,45sleep10-21swapper/122:20:411
223099540,6rtkit-daemon2229-21rtkit-daemon17:38:441
223099540,5rtkit-daemon2229-21rtkit-daemon20:05:161
223099540,5rtkit-daemon2229-21rtkit-daemon20:05:161
948099538,22cyclictest0-21swapper/221:05:262
948099535,27cyclictest0-21swapper/221:50:102
948099533,26cyclictest0-21swapper/219:15:162
947799533,45cyclictest0-21swapper/022:35:190
947799533,45cyclictest0-21swapper/019:25:170
947799533,45cyclictest0-21swapper/019:25:170
947799533,45cyclictest0-21swapper/018:45:130
947799533,44cyclictest0-21swapper/020:30:100
947799533,44cyclictest0-21swapper/019:45:120
9477995320,12cyclictest0-21swapper/022:50:340
9477995320,12cyclictest0-21swapper/022:50:340
64422530,3sleep36548-21timerandwakeup19:25:363
64422530,3sleep36548-21timerandwakeup19:25:363
223099530,5rtkit-daemon2229-21rtkit-daemon21:59:111
223099530,5rtkit-daemon2229-21rtkit-daemon21:00:281
948099525,28cyclictest0-21swapper/218:21:352
948099522,44cyclictest0-21swapper/219:55:232
9480995215,13cyclictest0-21swapper/218:06:322
9480995211,13cyclictest0-21swapper/218:36:542
947799525,40cyclictest0-21swapper/018:50:290
947799525,29cyclictest0-21swapper/018:31:370
947799523,43cyclictest0-21swapper/021:00:110
947799523,43cyclictest0-21swapper/019:10:160
947799522,44cyclictest0-21swapper/022:10:080
947799522,44cyclictest0-21swapper/022:10:080
70422524,40sleep10-21swapper/119:28:561
70422524,40sleep10-21swapper/119:28:561
223099520,5rtkit-daemon2229-21rtkit-daemon21:20:281
223099520,5rtkit-daemon2229-21rtkit-daemon20:15:301
223099520,5rtkit-daemon2229-21rtkit-daemon19:50:161
115452522,43sleep10-21swapper/117:56:021
115452522,43sleep10-21swapper/117:56:021
9481995137,10cyclictest0-21swapper/319:55:023
948199513,42cyclictest0-21swapper/317:40:313
948099514,25cyclictest0-21swapper/219:40:182
9479995135,11cyclictest0-21swapper/122:05:251
9479995135,10cyclictest1160-21snmpd22:57:221
947999513,41cyclictest0-21swapper/122:40:291
947799515,28cyclictest0-21swapper/018:36:380
947799515,27cyclictest0-21swapper/019:15:320
947799514,41cyclictest0-21swapper/017:45:070
947799513,43cyclictest0-21swapper/019:00:400
947799513,42cyclictest0-21swapper/018:06:320
284402512,43sleep10-21swapper/122:28:191
223099510,5rtkit-daemon2229-21rtkit-daemon22:35:171
223099510,5rtkit-daemon2229-21rtkit-daemon20:40:291
223099510,5rtkit-daemon2229-21rtkit-daemon18:55:181
223099510,5rtkit-daemon2229-21rtkit-daemon18:21:351
139582514,40sleep00-21swapper/018:16:120
9481995037,8cyclictest19969-21sshd21:01:203
9481995025,8cyclictest28650-21latency_hist20:15:023
9481995023,10cyclictest31634-21cpuspeed_turbos20:20:173
948099505,28cyclictest0-21swapper/218:27:322
9480995020,15cyclictest0-21swapper/218:01:312
9479995035,10cyclictest0-21swapper/122:50:221
9479995035,10cyclictest0-21swapper/122:50:221
9479995035,10cyclictest0-21swapper/121:15:301
9479995035,10cyclictest0-21swapper/120:00:261
9479995023,21cyclictest0-21swapper/118:15:541
947999502,8cyclictest20593-21taskset19:55:281
947799505,39cyclictest0-21swapper/019:54:560
947799505,26cyclictest0-21swapper/018:21:350
947799504,27cyclictest0-21swapper/017:56:300
947799504,27cyclictest0-21swapper/017:56:300
212902502,9sleep10-21swapper/122:13:321
212902502,9sleep10-21swapper/122:13:321
100552502,9sleep10-21swapper/117:45:061
9481994922,11cyclictest20090-21gltestperf22:10:163
9481994922,11cyclictest20090-21gltestperf22:10:163
947999498,34cyclictest0-21swapper/121:44:591
947999498,27cyclictest0-21swapper/118:30:061
947799495,16cyclictest0-21swapper/021:47:080
947799494,27cyclictest0-21swapper/018:01:310
223099490,5rtkit-daemon2229-21rtkit-daemon17:46:291
223099490,4rtkit-daemon2229-21rtkit-daemon20:45:291
223099490,4rtkit-daemon2229-21rtkit-daemon20:45:291
164422492,9sleep10-21swapper/122:02:321
153202493,9sleep10-21swapper/123:08:531
948199483,39cyclictest0-21swapper/322:17:293
9481994826,10cyclictest0-21swapper/319:05:163
9481994824,9cyclictest10630-21crond22:59:553
948099484,27cyclictest0-21swapper/217:56:302
948099484,27cyclictest0-21swapper/217:56:302
947999484,37cyclictest0-21swapper/118:10:081
9479994835,9cyclictest0-21swapper/119:20:351
9479994834,9cyclictest0-21swapper/119:15:151
9479994834,10cyclictest0-21swapper/121:50:301
9479994833,10cyclictest0-21swapper/119:05:391
9479994833,10cyclictest0-21swapper/118:50:171
947999483,41cyclictest0-21swapper/121:06:131
947999483,29cyclictest0-21swapper/119:00:421
44662482,40sleep30-21swapper/321:37:113
132842483,9sleep00-21swapper/018:10:120
948199474,36cyclictest0-21swapper/321:40:143
948199474,36cyclictest0-21swapper/319:30:343
948199473,37cyclictest0-21swapper/319:18:573
9481994725,10cyclictest0-21swapper/321:10:123
9481994724,7cyclictest16683-21latency_hist18:40:063
9479994733,9cyclictest12605-21unixbench_singl23:00:261
9479994733,9cyclictest1099-21munin-node21:30:191
9479994733,9cyclictest1099-21munin-node21:30:191
947799473,10cyclictest0-21swapper/017:45:190
223099470,5rtkit-daemon2229-21rtkit-daemon18:33:571
121612472,9sleep10-21swapper/118:01:141
948199464,35cyclictest0-21swapper/320:27:383
9481994633,9cyclictest0-21swapper/323:00:533
9481994632,9cyclictest0-21swapper/319:12:533
9481994624,8cyclictest14570-21munin-node19:45:273
9481994618,13cyclictest0-21swapper/319:40:033
9481994618,12cyclictest0-21swapper/320:45:013
9481994613,13cyclictest0-21swapper/318:50:313
948099465,13cyclictest0-21swapper/217:47:262
947999465,27cyclictest0-21swapper/121:25:101
9479994631,10cyclictest0-21swapper/120:20:291
947999463,37cyclictest0-21swapper/122:34:561
947999463,35cyclictest0-21swapper/117:55:081
9479994625,10cyclictest0-21swapper/120:51:381
9479994625,10cyclictest0-21swapper/119:39:361
57072462,38sleep30-21swapper/322:48:083
235282462,38sleep10-21swapper/122:16:381
181882462,9sleep10-21swapper/118:43:421
181882462,9sleep10-21swapper/118:43:421
153222464,37sleep30-21swapper/323:08:553
948199456,33cyclictest0-21swapper/322:00:193
948199455,33cyclictest0-21swapper/317:40:083
948199454,25cyclictest0-21swapper/321:52:093
9481994532,9cyclictest0-21swapper/321:23:063
948199453,36cyclictest0-21swapper/320:35:143
9481994524,8cyclictest2596-21munin-node19:20:233
9481994522,10cyclictest0-21swapper/321:30:063
9481994522,10cyclictest0-21swapper/321:30:063
9481994518,12cyclictest0-21swapper/320:30:243
9481994510,29cyclictest1584-21nscd18:25:053
948099454,27cyclictest0-21swapper/218:31:372
947999453,26cyclictest0-21swapper/122:45:071
9479994525,9cyclictest0-21swapper/118:12:561
271482452,36sleep30-21swapper/321:16:393
264242453,9sleep325342-21munin-node19:00:333
203232452,37sleep10-21swapper/118:50:031
948199445,33cyclictest0-21swapper/320:15:193
948199444,24cyclictest0-21swapper/321:05:173
948199444,15cyclictest0-21swapper/321:45:293
948199443,35cyclictest605-21sshd22:36:123
948199443,35cyclictest0-21swapper/320:50:183
948199443,35cyclictest0-21swapper/317:50:073
9481994429,9cyclictest30255-21python22:30:293
9481994427,11cyclictest0-21swapper/322:20:173
9481994426,12cyclictest0-21swapper/320:10:033
9481994426,12cyclictest0-21swapper/320:10:033
9481994424,7cyclictest21399-21chrt19:58:563
9481994416,12cyclictest0-21swapper/318:05:443
948099445,13cyclictest0-21swapper/217:51:292
9481994328,10cyclictest0-21swapper/318:26:483
9481994324,9cyclictest0-21swapper/322:50:303
9481994324,9cyclictest0-21swapper/322:50:303
9481994322,8cyclictest22211-21df_abs20:00:193
9481994319,9cyclictest3040-21unixbench_multi22:40:293
9481994315,13cyclictest0-21swapper/319:00:053
9480994324,12cyclictest1265-21runrttasks17:40:072
947999436,7cyclictest0-21swapper/119:30:271
947999434,13cyclictest0-21swapper/121:10:351
284102432,34sleep30-21swapper/322:27:573
9481994227,11cyclictest0-21swapper/317:59:503
9481994227,11cyclictest0-21swapper/317:59:503
9481994214,13cyclictest0-21swapper/320:45:223
9481994214,13cyclictest0-21swapper/320:45:223
168662422,33sleep30-21swapper/322:05:083
120832422,34sleep30-21swapper/318:00:113
948199414,30cyclictest16914-21munin-node18:40:413
948199414,30cyclictest16914-21munin-node18:40:413
948199413,24cyclictest31380-21processes21:25:273
9481994126,10cyclictest0-21swapper/317:53:263
9481994116,11cyclictest0-21swapper/318:20:063
9481994026,9cyclictest0-21swapper/318:12:203
9481994026,8cyclictest942-21crond18:31:063
9481994019,8cyclictest1160-21snmpd21:55:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional