You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 22:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Congatec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot8s.osadl.org (updated Wed Dec 24, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1756049911530,1147cyclictest242418-21sleep08:00:013
175604993810,377cyclictest377244-21turbostat09:49:553
175604992820,279cyclictest589638-21turbostat12:34:553
175604992623,1cyclictest0-21swapper/312:25:003
175604992421,1cyclictest0-21swapper/307:45:013
175604992017,1cyclictest0-21swapper/308:10:013
17560499176,7cyclictest4954122sleep311:15:183
175604991613,1cyclictest0-21swapper/311:50:433
175604991613,1cyclictest0-21swapper/310:20:183
175604991613,1cyclictest0-21swapper/310:15:003
175604991613,1cyclictest0-21swapper/309:55:153
175604991613,1cyclictest0-21swapper/309:25:183
175604991613,1cyclictest0-21swapper/309:19:153
175604991513,1cyclictest0-21swapper/308:53:383
175604991512,1cyclictest0-21swapper/312:11:503
175604991512,1cyclictest0-21swapper/312:00:503
175604991512,1cyclictest0-21swapper/311:40:113
175604991512,1cyclictest0-21swapper/311:40:103
175604991512,1cyclictest0-21swapper/310:48:093
175604991512,1cyclictest0-21swapper/310:25:223
175604991512,1cyclictest0-21swapper/310:05:183
175604991512,1cyclictest0-21swapper/309:39:333
175604991512,1cyclictest0-21swapper/308:25:233
17560499151,11cyclictest441713-21taskset10:35:153
17560499151,11cyclictest441433-21turbostat10:35:003
17560499150,12cyclictest390032-21turbostat09:55:003
2430562140,1sleep2243346-21sed08:00:152
17560499144,7cyclictest479710-21taskset11:01:233
175604991412,1cyclictest0-21swapper/311:23:123
175604991412,1cyclictest0-21swapper/310:57:243
175604991412,1cyclictest0-21swapper/309:07:063
175604991412,1cyclictest0-21swapper/308:17:163
175604991412,0cyclictest0-21swapper/311:08:223
175604991411,1cyclictest0-21swapper/311:25:153
175604991411,1cyclictest0-21swapper/309:44:073
175604991411,1cyclictest0-21swapper/308:44:163
175604991411,1cyclictest0-21swapper/307:45:283
17560499141,10cyclictest532036-21taskset11:45:143
17560499141,10cyclictest492788-21taskset11:14:573
17560499141,10cyclictest332402-21taskset09:10:113
17560499141,10cyclictest255438-21taskset08:10:093
17560499141,10cyclictest203919-21taskset07:30:143
17560499141,10cyclictest184423-21taskset07:15:133
17560499140,11cyclictest525153-21turbostat11:40:003
17560499140,11cyclictest281026-21taskset08:30:003
5838592130,7sleep2583456-21/usr/sbin/munin12:25:162
3838692132,7sleep1383902-21apt-get09:50:131
17560499131,9cyclictest5831162sleep312:25:023
17560499131,9cyclictest558994-21taskset12:05:173
17560499131,9cyclictest460995-21taskset10:50:123
17560499131,9cyclictest319524-21chrt09:00:023
17560499131,9cyclictest229655-21taskset07:50:023
175604991312,0cyclictest0-21swapper/308:21:203
175604991311,1cyclictest0-21swapper/308:37:453
175604991311,1cyclictest0-21swapper/307:20:233
17560499131,10cyclictest421988-21taskset10:17:353
17560499131,10cyclictest402856-21taskset10:04:563
175604991311,0cyclictest0-21swapper/311:30:223
175604991311,0cyclictest0-21swapper/307:35:563
17560499130,5cyclictest0-21swapper/307:10:013
17560499130,10cyclictest313114-21taskset08:55:013
5386832120,9sleep2538937-21cut11:50:152
5307822121,7sleep2530908-21sshd11:40:222
5307822121,7sleep2530908-21sshd11:40:212
4289042121,4sleep2429056-21df_inode10:25:122
3197332121,9sleep0319900-21gltestperf09:00:150
2441722120,9sleep0246186-21grep08:00:180
17560499129,1cyclictest0-21swapper/307:25:213
17560499121,9cyclictest602344-21taskset12:38:513
17560499121,9cyclictest306332-21taskset08:48:043
17560499121,9cyclictest243347-21taskset08:00:153
17560499121,8cyclictest570399-21chrt12:15:123
17560499121,4cyclictest0-21swapper/310:40:033
175604991210,1cyclictest0-21swapper/311:55:023
17560499120,9cyclictest358254-21chrt09:30:073
17560499120,10cyclictest345181-21taskset09:20:013
4794412110,3sleep2479626-21users11:00:272
4678152113,5sleep1468364-21irqrtprio10:55:161
301798110,3rtkit-daemon636-21snmpd10:00:351
301798110,3rtkit-daemon492-21gmain09:33:011
4788242100,7sleep0479065-21head11:00:200
3917252100,3sleep1393195-21cut09:55:191
301798100,6rtkit-daemon273775-21/usr/sbin/munin08:20:221
301798100,5rtkit-daemon508564-21kthreadcore11:25:182
301798100,5rtkit-daemon440582-21ntp_states10:30:221
301798100,5rtkit-daemon377668-21diskstats09:45:111
301798100,3rtkit-daemon636-21snmpd09:44:281
301798100,3rtkit-daemon492-21gmain08:26:411
301798100,1rtkit-daemon0-21swapper/211:33:282
2686442101,2sleep20-21swapper/208:20:162
582212291,4sleep1582323-21awk12:20:221
504749290,5sleep1504806-21ntp_states11:20:201
485767291,5sleep2486011-21users11:05:232
454940290,6sleep1455223-21sed10:45:141
427809291,4sleep1427926-21python310:20:231
421615291,6sleep0421808-21who10:15:240
383611291,5sleep0383614-21cron09:50:010
328064290,6sleep0330126-21cut09:05:170
30179890,5rtkit-daemon3205-21sendmail-mta11:47:362
30179890,5rtkit-daemon3205-21sendmail-mta09:17:071
30179890,5rtkit-daemon3205-21sendmail-mta09:05:121
30179890,5rtkit-daemon317-21systemd-udevd12:35:322
30179890,5rtkit-daemon317-21systemd-udevd08:07:331
30179890,5rtkit-daemon224627-21kthreadcore07:45:181
30179890,5rtkit-daemon182089-21munin-node09:00:151
30179890,4rtkit-daemon497570-21cut11:15:180
30179890,4rtkit-daemon448221-21cpuspeed_turbos10:40:130
30179890,4rtkit-daemon3016-21rtkit-daemon07:30:001
30179890,4rtkit-daemon236396-21diskstats07:55:151
30179890,3rtkit-daemon557726-21df_inode12:05:142
30179890,1rtkit-daemon0-21swapper/110:16:541
30179890,1rtkit-daemon0-21swapper/109:21:481
30179890,1rtkit-daemon0-21swapper/010:37:190
235947291,5sleep0235990-21munin-run07:55:000
184799290,6sleep2185503-21sed07:15:172
169367291,3sleep1169400-21apt-get07:05:141
545157280,6sleep0546021-21cut11:55:170
542011280,6sleep1538288-21/usr/sbin/munin11:50:181
539998281,4sleep0542008-21kthreadcore11:50:170
493060281,3sleep1493082-21apt-get11:15:121
492070281,4sleep1492179-21find11:10:231
487033281,4sleep2487358-21sed11:10:162
473553280,5sleep0473757-21latency_hist11:00:000
417965280,6sleep2420022-21kthreadcore10:15:182
337951281,4sleep2338157-21users09:10:232
30179880,5rtkit-daemon3205-21sendmail-mta07:50:061
30179880,4rtkit-daemon638-21ntpd07:30:141
30179880,4rtkit-daemon636-21snmpd12:30:382
30179880,4rtkit-daemon636-21snmpd11:06:240
30179880,4rtkit-daemon636-21snmpd08:55:071
30179880,4rtkit-daemon636-21snmpd08:34:571
30179880,4rtkit-daemon448-21wpa_supplicant10:53:040
30179880,4rtkit-daemon448-21wpa_supplicant08:38:171
30179880,4rtkit-daemon441710-21apt-get10:35:141
30179880,4rtkit-daemon3205-21sendmail-mta11:38:312
30179880,4rtkit-daemon3205-21sendmail-mta10:12:321
30179880,4rtkit-daemon3205-21sendmail-mta07:24:411
30179880,4rtkit-daemon2904-21cron08:41:001
30179880,4rtkit-daemon182089-21munin-node11:20:592
30179880,4rtkit-daemon182089-21munin-node10:49:020
30179880,4rtkit-daemon182089-21munin-node10:25:221
30179880,4rtkit-daemon182089-21munin-node08:02:151
30179880,4rtkit-daemon182089-21munin-node07:42:141
30179880,4rtkit-daemon182089-21munin-node07:17:001
30179880,4rtkit-daemon0-21swapper/108:50:221
30179880,3rtkit-daemon255474-21apt-get08:10:131
30179880,1rtkit-daemon0-21swapper/109:38:091
269978280,5sleep0271380-21kthreadcore08:20:170
261808281,4sleep0261839-21apt08:15:120
261551281,4sleep0261750-21cut08:15:010
596266270,3sleep1596298-21apt-get12:35:131
589871271,3sleep1589904-21apt-get12:30:131
578451270,4sleep0580436-21cut12:20:180
525415270,3sleep0525448-21apt-get11:40:120
525415270,3sleep0525448-21apt-get11:40:110
512225270,3sleep0512073-21apt-get11:30:020
466394271,3sleep1466503-21find10:50:241
461357271,3sleep2461674-21cut10:50:172
449602270,4sleep1451395-21cut10:40:171
433925270,4sleep0434105-21cut10:25:190
360201270,5sleep2362268-21cut09:30:202
350407271,4sleep2350486-21ntp_states09:20:212
337829270,4sleep1337950-21sort09:10:221
334407270,4sleep0336453-21tr09:10:170
30179870,4rtkit-daemon638-21ntpd12:00:332
30179870,4rtkit-daemon636-21snmpd08:45:291
30179870,4rtkit-daemon636-21snmpd08:17:211
30179870,4rtkit-daemon3205-21sendmail-mta09:27:461
30179870,3rtkit-daemon638-21ntpd11:56:112
30179870,3rtkit-daemon636-21snmpd12:15:182
30179870,3rtkit-daemon636-21snmpd11:12:110
30179870,3rtkit-daemon636-21snmpd07:35:171
30179870,3rtkit-daemon297-21systemd-journal10:06:081
30179870,3rtkit-daemon182089-21munin-node12:20:192
30179870,3rtkit-daemon182089-21munin-node12:10:202
286320271,3sleep0286397-21ntp_states08:30:540
233559270,4sleep2234911-21memory07:50:182
205217270,4sleep2207060-21cut07:30:192
197614270,4sleep0197781-21interrupts07:25:150
175779270,3sleep1175802-21apt-get07:10:101
175456270,1sleep30-21swapper/307:09:563
563954260,3sleep1563987-21apt-get12:10:131
544739260,3sleep1544885-21df11:55:141
532294261,2sleep10-21swapper/111:45:161
505447260,1sleep00-21swapper/011:22:280
494505260,3sleep2495410-21kthreadcore11:15:172
451397261,2sleep20-21swapper/210:40:182
446942260,3sleep2447013-21ntp_states10:35:232
410438261,2sleep2411342-21kthreadcore10:10:182
406011260,4sleep2408178-21cut10:05:182
396382260,1sleep00-21swapper/009:59:470
365074261,2sleep20-21swapper/209:35:192
358262260,3sleep0358295-21apt-get09:30:140
351267260,1sleep00-21swapper/009:24:160
340924261,2sleep00-21swapper/009:15:190
319242260,1sleep00-21swapper/008:59:470
299941260,1sleep00-21swapper/008:43:060
294107263,1sleep20-21swapper/208:40:142
293515260,1sleep00-21swapper/008:38:290
254153260,3sleep2254311-21ntpq08:05:202
227159260,3sleep0228213-21memory07:45:200
223171260,3sleep2223258-21cut07:45:132
191091261,3sleep2191204-21/usr/sbin/munin07:20:162
173737260,2sleep00-21swapper/007:05:210
170442260,3sleep2171638-21cat07:05:182
595779250,1sleep00-21swapper/012:32:040
589612251,2sleep11-21systemd12:30:001
531334250,1sleep10-21swapper/111:41:371
531334250,1sleep10-21swapper/111:41:361
519029250,3sleep0519184-21diskstats11:35:140
518602250,1sleep10-21swapper/111:33:191
507358251,2sleep10-21swapper/111:25:171
505900250,2sleep00-21swapper/011:25:120
479644250,1sleep10-21swapper/111:00:341
473218250,1sleep20-21swapper/210:55:412
460594250,1sleep20-21swapper/210:48:372
441353250,1sleep00-21swapper/010:34:110
435684251,2sleep20-21swapper/210:30:172
415501250,1sleep00-21swapper/010:12:010
403116250,2sleep00-21swapper/010:05:100
402862250,2sleep20-21swapper/210:05:002
397084251,2sleep00-21swapper/010:00:180
390002250,1sleep20-21swapper/209:54:552
377039250,1sleep20-21swapper/209:42:422
374686251,2sleep00-21swapper/009:40:180
351846250,2sleep0351879-21apt-get09:25:120
339183251,2sleep20-21swapper/209:15:172
325658250,1sleep20-21swapper/209:04:352
306933251,2sleep00-21swapper/008:50:150
306159250,1sleep20-21swapper/208:45:512
287113250,1sleep20-21swapper/208:33:572
280613250,1sleep00-21swapper/008:27:510
222823250,1sleep00-21swapper/007:44:190
222626250,1sleep20-21swapper/207:41:452
215507250,3sleep20-21swapper/207:35:232
215353250,2sleep00-21swapper/007:35:210
190900250,2sleep00-21swapper/007:20:150
181615250,1sleep00-21swapper/007:10:530
181094251,2sleep20-21swapper/207:10:192
585531240,2sleep0587588-21kthreadcore12:25:180
572584240,2sleep00-21swapper/012:15:180
569914240,3sleep00-21swapper/012:12:280
557281240,2sleep1557300-21munin-run12:05:001
428325240,3sleep20-21swapper/210:21:202
423910241,1sleep00-21swapper/010:20:170
383538241,2sleep20-21swapper/209:49:062
325947240,1sleep20-21swapper/209:05:102
313325240,2sleep20-21swapper/208:55:172
306371240,3sleep00-21swapper/008:48:330
290609240,2sleep20-21swapper/208:35:182
274646240,1sleep20-21swapper/208:25:112
261478240,3sleep20-21swapper/208:14:072
238491240,2sleep20-21swapper/207:55:192
209727240,3sleep00-21swapper/007:32:540
203244240,3sleep20-21swapper/207:27:102
186660240,2sleep0188724-21kthreadcore07:15:180
601756230,1sleep00-21swapper/012:35:240
575827230,1sleep10-21swapper/112:15:231
562595230,1sleep1562767-21strings12:05:211
557530230,1sleep00-21swapper/012:05:080
551117230,1sleep00-21swapper/012:00:060
519185230,1sleep10-21swapper/111:35:151
484101230,1sleep10-21swapper/111:05:181
395609230,1sleep20-21swapper/209:55:242
389672230,1sleep00-21swapper/009:50:360
364816230,1sleep00-21swapper/009:35:170
353730230,1sleep20-21swapper/209:25:182
311974230,1sleep20-21swapper/208:50:212
261888230,1sleep20-21swapper/208:15:142
251235230,1sleep00-21swapper/008:05:180
236203230,1sleep00-21swapper/007:55:130
531841220,1sleep00-21swapper/011:45:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional