You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 17:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot2.osadl.org (updated Tue Jul 14, 2026 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22312169917186,77cyclictest0-21swapper/011:30:000
22312169917186,77cyclictest0-21swapper/011:30:000
223121899167154,6cyclictest79-21hwrng10:27:302
223121799167156,4cyclictest0-21swapper/108:05:181
22312189915130,55cyclictest2403136-21ls12:10:002
22312189915130,55cyclictest2403136-21ls12:10:002
223121699147110,29cyclictest2323275-21ssh10:15:560
223121999139135,2cyclictest0-21swapper/309:15:503
223121999139135,2cyclictest0-21swapper/309:15:503
223121899133119,7cyclictest79-21hwrng12:16:032
2231219991323,92cyclictest2421406-21perl12:35:343
223121999130118,8cyclictest79-21hwrng12:32:263
22312179913056,64cyclictest10950irq/205-8000000.ethernet-rx011:20:521
223121699129114,12cyclictest79-21hwrng09:32:120
22312189912858,11cyclictest0-21swapper/209:10:142
22312189912710,83cyclictest2317359-21kworker/u16:1+rpciod10:21:082
2231217991263,25cyclictest2355144-21sleep11:01:171
22312189912549,31cyclictest0-21swapper/209:00:382
2231217991255,114cyclictest2366249-21users11:16:301
22312199912435,79cyclictest0-21swapper/310:41:083
22312199912361,21cyclictest0-21swapper/311:20:363
22312179912336,79cyclictest301ktimers/109:05:161
223121899122106,6cyclictest79-21hwrng09:48:352
22312189912083,34cyclictest79-21hwrng11:27:552
22312189912083,34cyclictest79-21hwrng11:27:552
223121899120101,7cyclictest79-21hwrng10:16:142
223121799120105,12cyclictest10950irq/205-8000000.ethernet-rx009:40:181
223121999119103,13cyclictest10950irq/205-8000000.ethernet-rx010:00:243
223121799119110,5cyclictest79-21hwrng10:00:531
2231217991181,92cyclictest2408974-21kworker/u16:4+rpciod12:40:011
223121999117107,8cyclictest461ktimers/309:28:013
223121899117105,8cyclictest79-21hwrng09:18:542
223121899117105,8cyclictest79-21hwrng09:18:542
223121899116108,4cyclictest79-21hwrng12:39:362
223121799116103,9cyclictest79-21hwrng08:15:241
22312199911599,6cyclictest79-21hwrng10:29:333
223121999115104,6cyclictest79-21hwrng09:10:423
2231217991153,33cyclictest2366891-21kworker/u16:9+rpciod12:30:001
223121999114107,5cyclictest79-21hwrng07:44:413
223121999114107,5cyclictest79-21hwrng07:44:413
223121699114106,6cyclictest79-21hwrng09:02:310
223121899113102,9cyclictest10950irq/205-8000000.ethernet-rx010:01:022
223121899113100,7cyclictest79-21hwrng11:43:172
2231217991137,11cyclictest2276454-21apt-get09:10:271
2231217991134,86cyclictest56-21kcompactd011:30:311
2231219991127,71cyclictest2333310-21grep10:30:363
22312189911294,12cyclictest0-21swapper/212:25:592
223121899112104,5cyclictest79-21hwrng09:26:042
223121799112101,7cyclictest79-21hwrng07:50:501
22312199911198,11cyclictest79-21hwrng12:15:013
22312199911196,5cyclictest79-21hwrng08:39:593
22312189911131,5cyclictest381ktimers/209:50:122
22312179911124,43cyclictest19-21rcu_preempt12:10:161
223121699111104,5cyclictest79-21hwrng11:33:020
22312199911085,18cyclictest461ktimers/307:10:553
2231217991108,95cyclictest2360944-21ssh11:10:161
2231217991108,95cyclictest2360944-21ssh11:10:161
223121799110100,5cyclictest79-21hwrng12:30:231
223121699110103,5cyclictest79-21hwrng11:44:180
223121999109100,6cyclictest79-21hwrng08:10:173
223121999109100,6cyclictest79-21hwrng08:10:173
223121899109102,5cyclictest79-21hwrng11:14:362
223121899109102,5cyclictest79-21hwrng11:14:362
22312179910997,4cyclictest79-21hwrng08:46:081
22312179910997,4cyclictest79-21hwrng08:46:081
2231216991098,99cyclictest0-21swapper/009:10:320
223121999108100,5cyclictest79-21hwrng08:41:003
2231219991074,84cyclictest0-21swapper/312:20:543
22312169910771,4cyclictest10350irq/131-8000000.ethernet-tx208:00:480
22312199910695,8cyclictest0-21swapper/312:01:133
22312199910637,52cyclictest2325467-21sendmail-msp10:20:193
22312199910546,17cyclictest461ktimers/308:50:563
22312199910534,30cyclictest2352603-21kworker/u16:8+rpciod11:40:143
22312189910595,6cyclictest79-21hwrng10:50:022
22312189910564,18cyclictest381ktimers/208:55:132
22312179910597,5cyclictest79-21hwrng09:35:171
223121799105101,2cyclictest0-21swapper/110:35:481
223121799105101,2cyclictest0-21swapper/110:35:481
22312189910461,38cyclictest381ktimers/209:20:352
22312179910474,27cyclictest301ktimers/111:25:211
22312179910474,27cyclictest301ktimers/111:25:211
22312169910480,17cyclictest21-21rcuc/010:06:060
22312199910293,5cyclictest79-21hwrng07:35:283
22312199910293,5cyclictest79-21hwrng07:35:283
22312199910292,7cyclictest79-21hwrng10:51:033
22312199910286,6cyclictest79-21hwrng07:25:143
22312169910270,27cyclictest0-21swapper/009:21:330
22312199910140,23cyclictest0-21swapper/311:10:283
22312199910140,23cyclictest0-21swapper/311:10:283
22312189910141,24cyclictest2344275-21ntpq10:45:442
22312199910093,5cyclictest79-21hwrng09:58:503
22312199910088,8cyclictest79-21hwrng07:16:013
22312199910088,8cyclictest79-21hwrng07:16:013
22312199910079,18cyclictest10950irq/205-8000000.ethernet-rx009:50:453
22312199910048,15cyclictest2364553-21ssh11:15:273
22312199910029,28cyclictest2243518-21kworker/u16:4+rpciod09:00:153
22312189910073,23cyclictest0-21swapper/208:41:022
2231219999988,7cyclictest79-21hwrng08:00:033
2231216999931,64cyclictest21-21rcuc/012:05:580
2231216999931,64cyclictest21-21rcuc/012:05:580
2231219999810,48cyclictest2390221-21ntpq11:51:023
2231218999890,5cyclictest79-21hwrng11:04:222
223121799984,32cyclictest2390409-21ssh11:51:201
2231219999714,14cyclictest45-21rcuc/311:00:063
2231218999791,4cyclictest0-21swapper/207:35:202
2231218999791,4cyclictest0-21swapper/207:35:202
2231217999784,8cyclictest79-21hwrng10:14:111
2231217999784,8cyclictest79-21hwrng10:14:111
223121799973,72cyclictest2387689-21sleep11:47:111
223121999965,45cyclictest2385959-21df11:45:303
2231218999688,6cyclictest79-21hwrng09:57:482
2231218999681,11cyclictest79-21hwrng11:56:352
2231217999683,10cyclictest79-21hwrng10:19:191
2231217999677,17cyclictest10950irq/205-8000000.ethernet-rx010:20:291
2231217999625,4cyclictest0-21swapper/109:15:101
2231217999625,4cyclictest0-21swapper/109:15:101
2231216999688,5cyclictest79-21hwrng09:41:250
223121699962,20cyclictest2363294-21kworker/u16:2+xprtiod11:15:000
223121699962,20cyclictest2363294-21kworker/u16:2+xprtiod11:15:000
2231219999534,28cyclictest461ktimers/309:21:223
2231218999585,3cyclictest0-21swapper/210:41:032
2231217999515,17cyclictest0-21swapper/112:01:041
223121899945,6cyclictest2350365-21apt-get10:55:292
223121899945,6cyclictest2350365-21apt-get10:55:292
223121699942,59cyclictest16-21ksoftirqd/010:55:210
223121699942,59cyclictest16-21ksoftirqd/010:55:210
2231219999338,26cyclictest0-21swapper/312:00:013
2231218999385,5cyclictest79-21hwrng11:24:512
2231219999279,10cyclictest79-21hwrng10:06:003
223121799928,39cyclictest2304369-21latency_hist09:49:581
2231217999240,46cyclictest301ktimers/112:16:021
2231217999231,38cyclictest2249723-21kworker/u16:2+rpciod08:40:091
2231216999281,9cyclictest0-21swapper/010:10:180
2231216999281,9cyclictest0-21swapper/010:10:180
2231219999182,6cyclictest79-21hwrng11:07:263
2231218999140,16cyclictest10950irq/205-8000000.ethernet-rx012:20:262
2231217999182,5cyclictest79-21hwrng10:32:371
2231217999130,49cyclictest2410053-21sendmail-msp12:20:371
2231216999184,5cyclictest171ktimers/008:15:420
2231219999083,5cyclictest79-21hwrng12:26:173
223121799906,29cyclictest2379598-21ntpq11:35:561
2231217999056,28cyclictest0-21swapper/109:20:241
2231217999012,11cyclictest10950irq/205-8000000.ethernet-rx009:50:391
2231216999030,32cyclictest2303468-21kworker/u16:0+rpciod10:00:080
2231219998980,5cyclictest0-21swapper/312:10:463
2231219998940,45cyclictest0-21swapper/310:16:153
2231217998980,5cyclictest79-21hwrng07:20:071
2231217998911,13cyclictest2339833-21apt-get10:40:311
2231219998875,4cyclictest79-21hwrng07:51:513
2231217998828,49cyclictest0-21swapper/107:40:101
2231217998828,49cyclictest0-21swapper/107:40:101
223121799882,63cyclictest0-21swapper/108:00:521
2231216998855,25cyclictest2338752-21kworker/u16:7+rpciod10:41:130
2231219998736,26cyclictest461ktimers/307:30:563
2231219998736,26cyclictest461ktimers/307:30:563
2231217998757,25cyclictest0-21swapper/107:10:331
223121799871,70cyclictest2381807-21sendmail-msp11:40:231
223121799871,15cyclictest2290705-21aten_r8power_en09:30:281
2231216998760,18cyclictest19-21rcu_preempt07:35:340
2231216998760,18cyclictest19-21rcu_preempt07:35:340
2231216998745,12cyclictest0-21swapper/011:20:120
2231219998668,14cyclictest79-21hwrng07:58:003
2231217998651,22cyclictest301ktimers/110:50:141
2231217998637,36cyclictest2257463-21sendmail-msp08:20:391
2231216998631,20cyclictest2412028-21unixbench_multi12:21:260
2231218998561,18cyclictest0-21swapper/211:15:122
2231218998557,21cyclictest381ktimers/207:25:442
2231217998525,4cyclictest0-21swapper/108:25:221
2231217998525,4cyclictest0-21swapper/108:25:221
2231219998477,4cyclictest79-21hwrng08:22:343
2231218998476,5cyclictest79-21hwrng09:42:272
2231217998430,51cyclictest301ktimers/107:35:551
2231217998430,51cyclictest301ktimers/107:35:551
223121799841,77cyclictest2245370-21ntpq07:45:501
2231216998457,22cyclictest171ktimers/008:50:130
2231216998443,23cyclictest2366891-21kworker/u16:9+xprtiod12:15:140
2231216998414,59cyclictest16-21ksoftirqd/009:36:590
2231218998373,8cyclictest79-21hwrng11:37:082
223121799834,22cyclictest2272636-21/usr/sbin/munin09:00:381
223121799833,34cyclictest2248293-21apt-get07:55:111
223121799832,27cyclictest2310106-21ssh09:56:511
2231216998373,4cyclictest171ktimers/009:55:570
2231216998362,18cyclictest0-21swapper/009:20:020
2231216998362,18cyclictest0-21swapper/009:20:020
2231216998357,13cyclictest0-21swapper/012:00:220
2231216998341,37cyclictest0-21swapper/007:40:550
2231216998341,37cyclictest0-21swapper/007:40:550
2231218998256,21cyclictest0-21swapper/210:05:102
2231217998236,41cyclictest2316188-21ssh10:05:501
2231217998223,6cyclictest0-21swapper/107:31:061
2231217998223,6cyclictest0-21swapper/107:31:061
2231217998118,26cyclictest2344476-21ssh10:46:001
223121799811,30cyclictest2393521-21ssh11:55:421
223121799811,29cyclictest2286996-21/usr/sbin/munin09:26:161
2231216998140,29cyclictest2349012-21kworker/u16:2+xprtiod11:01:240
2231216998131,23cyclictest171ktimers/010:35:180
2231216998131,23cyclictest171ktimers/010:35:180
2231216998113,37cyclictest0-21swapper/007:15:000
2231219998074,4cyclictest2351544-21ntpq10:56:073
2231219998074,4cyclictest2351544-21ntpq10:56:073
2231218998063,10cyclictest79-21hwrng08:21:332
223121799801,31cyclictest2353640-21ssh11:00:021
223121799801,31cyclictest2353640-21ssh11:00:021
223121799801,27cyclictest2332336-21ssh10:29:361
223121799801,25cyclictest2233706-21/usr/sbin/munin07:15:341
223121799801,25cyclictest2233706-21/usr/sbin/munin07:15:341
2231216998047,20cyclictest0-21swapper/010:20:590
2231219997972,4cyclictest79-21hwrng08:32:493
223121799791,27cyclictest2401893-21sshd-auth12:07:221
223121799791,27cyclictest2401893-21sshd-auth12:07:221
2231216997959,18cyclictest21-21rcuc/012:25:590
2231216997953,14cyclictest171ktimers/009:25:210
2231216997935,35cyclictest0-21swapper/011:35:110
2231218997869,7cyclictest0-21swapper/209:30:472
2231216997857,12cyclictest171ktimers/007:45:500
223121699784,43cyclictest16-21ksoftirqd/010:25:050
223121699781,60cyclictest0-21swapper/011:50:240
2231219997748,24cyclictest0-21swapper/309:40:423
2231218997744,29cyclictest0-21swapper/207:20:102
2231217997729,41cyclictest301ktimers/108:55:491
223121799771,29cyclictest2359911-21ssh11:08:161
2231219997645,21cyclictest2259112-21kworker/u16:11+xprtiod09:40:003
2231218997625,44cyclictest2274087-21kworker/u16:10+rpciod09:10:002
223121799762,32cyclictest2262310-21ntpq08:31:011
223121799761,29cyclictest2264099-21ntpq08:35:491
223121799761,29cyclictest2237643-21/usr/sbin/munin07:25:391
223121799761,27cyclictest2254722-21/usr/sbin/munin08:10:481
223121799761,27cyclictest2254722-21/usr/sbin/munin08:10:481
2231216997628,39cyclictest16-21ksoftirqd/010:54:430
223121699762,26cyclictest2408974-21kworker/u16:4+rpciod12:40:000
2231219997555,16cyclictest0-21swapper/309:45:413
2231216997566,3cyclictest0-21swapper/011:45:490
223121699754,67cyclictest2301994-21ssh09:45:410
223121699752,66cyclictest16-21ksoftirqd/010:45:240
2231219997451,19cyclictest0-21swapper/311:30:493
2231219997451,19cyclictest0-21swapper/308:16:083
2231219997446,24cyclictest0-21swapper/309:30:353
223121799742,26cyclictest2269061-21/usr/sbin/munin08:50:281
2231218997349,20cyclictest0-21swapper/211:30:152
2231218997335,32cyclictest0-21swapper/211:05:212
2231216997362,6cyclictest0-21swapper/012:10:500
2231216997351,17cyclictest171ktimers/007:30:120
2231216997351,17cyclictest171ktimers/007:30:120
223121699733,57cyclictest16-21ksoftirqd/010:33:000
2231219997248,16cyclictest0-21swapper/307:46:023
2231218997264,5cyclictest79-21hwrng09:37:202
2231216997257,11cyclictest0-21swapper/007:20:100
2231216997140,14cyclictest21-21rcuc/011:15:110
2231216997116,29cyclictest2025639-21kworker/u16:4+rpciod07:20:000
2231216997116,29cyclictest2025639-21kworker/u16:4+rpciod07:20:000
2231219997053,12cyclictest0-21swapper/310:46:003
2231219997047,14cyclictest461ktimers/308:25:143
2231219997047,14cyclictest461ktimers/308:25:143
223121699707,33cyclictest16-21ksoftirqd/011:06:210
2231216997051,10cyclictest0-21swapper/008:20:110
2231216997047,20cyclictest171ktimers/008:45:130
2231216997047,20cyclictest171ktimers/008:45:130
2231216997041,19cyclictest0-21swapper/009:06:070
223121899697,47cyclictest39-21ksoftirqd/211:50:372
2231218996961,6cyclictest79-21hwrng11:49:252
2231218996950,11cyclictest0-21swapper/212:00:492
2231219996855,9cyclictest0-21swapper/311:30:003
2231219996855,9cyclictest0-21swapper/311:30:003
2231218996849,16cyclictest0-21swapper/207:40:142
2231218996849,16cyclictest0-21swapper/207:40:142
2231216996855,11cyclictest171ktimers/007:50:140
2231216996840,19cyclictest0-21swapper/008:25:350
2231216996840,19cyclictest0-21swapper/008:25:350
2231216996839,20cyclictest0-21swapper/012:30:140
2231216996835,31cyclictest0-21swapper/008:59:590
2231216996828,23cyclictest2363295-21kworker/u16:7+rpciod12:00:010
2231218996747,11cyclictest0-21swapper/210:34:572
223121699662,47cyclictest2304624-21ssh09:50:120
2231218996533,18cyclictest2337412-21sleep10:35:582
2231218996533,18cyclictest2337412-21sleep10:35:582
223121899651,30cyclictest2404934-21/usr/sbin/munin12:11:132
2231216996543,18cyclictest0-21swapper/007:25:350
2231219996436,23cyclictest1-21systemd08:55:543
223121999632,29cyclictest10950irq/205-8000000.ethernet-rx010:11:023
223121999632,29cyclictest10950irq/205-8000000.ethernet-rx010:11:023
2231218996353,8cyclictest2231215-21cyclictest08:35:392
2231218996346,13cyclictest0-21swapper/207:50:292
2231218996333,16cyclictest1-21systemd07:10:192
2231218996327,26cyclictest2319763-21ssh10:10:532
2231218996327,26cyclictest2319763-21ssh10:10:532
2231216996330,31cyclictest2248818-21sh07:56:220
2231219996237,20cyclictest461ktimers/310:35:113
2231219996237,20cyclictest461ktimers/310:35:113
2231218996242,12cyclictest0-21swapper/208:01:022
2231218996240,10cyclictest0-21swapper/207:20:002
2231218996240,10cyclictest0-21swapper/207:20:002
2231218996229,29cyclictest2256639-21ntpq08:15:542
2231219996138,9cyclictest0-21swapper/307:20:133
2231218996125,32cyclictest2254788-21ntpq08:10:562
2231218996125,32cyclictest2254788-21ntpq08:10:562
2231216996147,10cyclictest0-21swapper/008:31:080
2231219996049,9cyclictest79-21hwrng08:49:123
2231219996049,9cyclictest79-21hwrng08:49:123
2231219996045,12cyclictest79-21hwrng09:09:413
223121799601,26cyclictest2231307-21latency_hist07:10:001
2231216996021,32cyclictest21-21rcuc/008:05:370
2231218995942,13cyclictest0-21swapper/208:49:592
2231218995942,13cyclictest0-21swapper/208:49:592
2231216995940,16cyclictest0-21swapper/008:40:310
2231218995828,26cyclictest2260460-21awk08:26:032
2231218995828,26cyclictest2260460-21awk08:26:032
2231216995834,20cyclictest2231215-21cyclictest08:35:460
2231219995739,10cyclictest0-21swapper/312:05:433
2231219995739,10cyclictest0-21swapper/312:05:433
223121899573,36cyclictest37-21rcuc/208:50:212
2231218995635,11cyclictest0-21swapper/212:30:282
2231218995629,23cyclictest2248564-21df_inode07:55:302
2231216995639,9cyclictest0-21swapper/008:10:330
2231216995639,9cyclictest0-21swapper/008:10:330
223121999551,24cyclictest2381958-21latency_hist11:40:023
2231218995427,23cyclictest2252330-21df_inode08:05:302
223121899541,44cyclictest2245420-21ntpq07:45:562
2231219995345,5cyclictest79-21hwrng08:09:163
2231218995328,18cyclictest2239704-21ntpq07:30:432
2231218995328,18cyclictest2239704-21ntpq07:30:432
2231218995320,22cyclictest645-21snmpd08:30:102
2231216995030,7cyclictest0-21swapper/007:10:000
2231219994936,9cyclictest0-21swapper/307:09:593
2231218994331,9cyclictest2231361-21latency_hist07:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional