You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-10 - 09:45
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack8slot2.osadl.org (updated Mon Aug 10, 2026 00:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26871339915921,8cyclictest0-21swapper/223:20:372
268713199157103,51cyclictest171ktimers/022:50:150
26871359914970,75cyclictest2792779-21ntpq22:35:553
268713399148110,29cyclictest0-21swapper/223:01:032
268713399148110,29cyclictest0-21swapper/223:01:032
268713399143131,9cyclictest2827601-21open_files23:26:022
268713399143131,9cyclictest2827601-21open_files23:26:022
268713199139103,30cyclictest0-21swapper/020:00:210
26871339913632,33cyclictest2741347-21kworker/u16:1+xprtiod22:01:152
268713599134110,20cyclictest0-21swapper/300:35:263
268713599131114,7cyclictest0-21swapper/323:20:483
268713599131114,12cyclictest79-21hwrng19:20:503
268713399130116,6cyclictest79-21hwrng00:11:392
26871359912937,64cyclictest0-21swapper/321:21:443
268713399129115,6cyclictest79-21hwrng22:25:092
2687133991281,67cyclictest2798964-21df_inode22:45:282
2687133991281,67cyclictest2798964-21df_inode22:45:282
268713299128114,4cyclictest79-21hwrng21:16:331
268713299128114,4cyclictest79-21hwrng21:16:331
268713399126117,5cyclictest79-21hwrng21:58:322
268713399126117,5cyclictest79-21hwrng21:58:322
268713399126117,5cyclictest79-21hwrng21:58:322
268713399124102,15cyclictest0-21swapper/220:00:432
268713599123113,7cyclictest0-21swapper/320:21:003
268713399121105,12cyclictest79-21hwrng19:47:282
268713399120113,4cyclictest79-21hwrng00:21:532
268713399120110,6cyclictest79-21hwrng22:07:452
268713399120110,6cyclictest79-21hwrng22:07:452
268713399120105,10cyclictest79-21hwrng00:06:322
268713399120105,10cyclictest79-21hwrng00:06:322
26871359911921,19cyclictest2715064-21kworker/u16:15+xprtiod21:09:573
26871359911921,19cyclictest2715064-21kworker/u16:15+xprtiod21:09:573
268713599119108,7cyclictest79-21hwrng21:01:113
268713599119108,7cyclictest79-21hwrng21:01:113
26871339911994,19cyclictest0-21swapper/219:21:012
268713399119111,5cyclictest79-21hwrng00:04:292
26871359911848,50cyclictest461ktimers/319:15:253
268713599118107,6cyclictest79-21hwrng22:30:173
268713299118109,6cyclictest79-21hwrng23:21:281
268713599117107,6cyclictest79-21hwrng19:10:363
268713199116106,6cyclictest79-21hwrng20:31:300
268713399115101,11cyclictest79-21hwrng19:15:432
268713399115100,6cyclictest79-21hwrng23:07:082
268713299115107,5cyclictest79-21hwrng19:59:451
268713399114107,4cyclictest79-21hwrng00:27:012
268713399114105,5cyclictest79-21hwrng20:26:222
268713299114106,5cyclictest79-21hwrng20:22:171
268713199114106,6cyclictest79-21hwrng19:51:330
26871359911393,14cyclictest79-21hwrng21:55:283
26871359911393,14cyclictest79-21hwrng21:55:283
26871359911393,14cyclictest79-21hwrng21:55:283
2687135991134,87cyclictest2775204-21ntpq22:10:403
2687133991131,93cyclictest2778578-21memory22:15:352
2687135991121,47cyclictest2810377-21/usr/sbin/munin23:01:173
2687135991121,47cyclictest2810377-21/usr/sbin/munin23:01:173
2687135991121,21cyclictest2861232-21apt-get00:15:063
268713599112101,7cyclictest79-21hwrng22:20:023
268713599112101,7cyclictest79-21hwrng22:20:023
268713399112103,6cyclictest79-21hwrng21:10:242
26871319911299,10cyclictest79-21hwrng19:33:070
268713199112104,5cyclictest79-21hwrng20:19:120
26871359911185,21cyclictest0-21swapper/323:41:203
26871359911185,21cyclictest0-21swapper/323:41:203
268713399111104,5cyclictest79-21hwrng21:53:252
2687133991102,70cyclictest2728344-21sendmail-msp21:00:132
2687133991102,70cyclictest2728344-21sendmail-msp21:00:132
26871359910996,10cyclictest2822691-21sendmail23:19:543
26871359910996,10cyclictest2822691-21sendmail23:19:543
268713399109101,5cyclictest79-21hwrng22:54:512
26871339910845,47cyclictest0-21swapper/220:41:072
26871339910845,47cyclictest0-21swapper/220:41:072
26871329910897,9cyclictest79-21hwrng20:38:401
26871359910739,35cyclictest2760923-21cut21:50:343
268713399107100,5cyclictest79-21hwrng00:33:092
268713399107100,3cyclictest0-21swapper/222:30:462
26871329910799,5cyclictest79-21hwrng19:21:521
26871319910798,6cyclictest79-21hwrng21:26:470
26871359910658,38cyclictest2850519-21sendmail-msp00:01:013
26871359910636,63cyclictest2858673-21ssh00:10:403
26871339910697,6cyclictest79-21hwrng19:50:322
26871319910698,4cyclictest79-21hwrng21:02:130
26871319910698,4cyclictest79-21hwrng21:02:130
26871359910511,18cyclictest2753934-21diskstats21:40:413
26871339910598,5cyclictest79-21hwrng23:38:532
26871339910589,6cyclictest79-21hwrng22:43:352
2687135991046,45cyclictest2715466-21apt-get20:25:093
26871329910442,29cyclictest2873243-21grep00:31:071
26871319910490,10cyclictest79-21hwrng19:39:160
2687131991041,89cyclictest2792586-21ssh22:35:400
2687135991031,72cyclictest2855513-21/usr/sbin/munin00:06:073
2687135991031,72cyclictest2855513-21/usr/sbin/munin00:06:073
26871339910360,18cyclictest2774824-21/usr/sbin/munin22:10:322
26871319910395,6cyclictest79-21hwrng19:26:590
26871319910395,6cyclictest79-21hwrng19:26:590
26871319910394,5cyclictest79-21hwrng20:08:580
26871359910294,5cyclictest79-21hwrng21:45:133
26871359910294,5cyclictest79-21hwrng21:45:133
26871359910231,61cyclictest0-21swapper/322:55:103
26871339910295,5cyclictest79-21hwrng23:54:152
26871339910293,5cyclictest79-21hwrng23:31:432
2687133991021,66cyclictest2758620-21sh21:46:102
2687133991021,66cyclictest2758620-21sh21:46:102
26871329910292,8cyclictest0-21swapper/100:20:461
26871319910280,16cyclictest0-21swapper/000:20:530
26871339910187,5cyclictest79-21hwrng00:36:142
26871359910032,31cyclictest2710119-21fschecks_count20:10:333
26871359910032,31cyclictest2710119-21fschecks_count20:10:333
26871339910093,5cyclictest79-21hwrng21:17:342
26871339910093,5cyclictest79-21hwrng21:17:342
2687132991002,93cyclictest0-21swapper/121:31:021
2687132991002,93cyclictest0-21swapper/121:31:021
2687135999989,7cyclictest79-21hwrng19:45:253
268713299993,54cyclictest2685169-21kworker/u16:10+rpciod20:00:261
2687135999889,5cyclictest79-21hwrng23:45:023
2687135999889,5cyclictest79-21hwrng23:45:023
2687135999888,7cyclictest2871777-21http00:30:233
2687135999854,40cyclictest461ktimers/320:00:303
2687135999840,46cyclictest2733265-21ssh21:10:423
2687132999880,15cyclictest29-21rcuc/123:25:361
2687132999880,15cyclictest29-21rcuc/123:25:361
2687135999783,8cyclictest79-21hwrng22:29:153
2687135999724,18cyclictest0-21swapper/322:50:313
2687135999713,15cyclictest2865914-21ntpq00:21:063
2687133999791,4cyclictest79-21hwrng19:25:572
2687133999791,4cyclictest79-21hwrng19:25:572
2687133999786,7cyclictest79-21hwrng22:56:542
2687132999770,13cyclictest0-21swapper/121:40:111
2687131999788,6cyclictest79-21hwrng22:33:210
268713599969,24cyclictest2844788-21ntpq23:50:423
2687133999689,5cyclictest79-21hwrng20:06:552
268713299961,5cyclictest2811066-21kworker/u16:10+rpciod23:19:561
268713299961,5cyclictest2811066-21kworker/u16:10+rpciod23:19:561
2687131999681,11cyclictest0-21swapper/023:20:590
2687135999536,44cyclictest0-21swapper/322:05:313
2687135999536,44cyclictest0-21swapper/322:05:313
2687131999571,19cyclictest171ktimers/021:45:570
2687131999571,19cyclictest171ktimers/021:45:570
2687131999485,6cyclictest79-21hwrng20:14:050
2687131999485,6cyclictest79-21hwrng20:14:050
2687131999477,6cyclictest21-21rcuc/000:39:570
2687135999325,25cyclictest387-21dbus-broker19:29:553
2687135999325,25cyclictest387-21dbus-broker19:29:553
2687131999388,3cyclictest171ktimers/021:50:150
268713199931,25cyclictest2729978-21kworker/u16:17+rpciod21:19:570
2687133999277,10cyclictest79-21hwrng19:13:402
2687132999285,4cyclictest2724602-21latency_hist20:49:551
268713599914,64cyclictest1123-21rs:main3
2687135999126,54cyclictest0-21swapper/321:29:563
2687135999122,26cyclictest2767103-21sendmail-msp22:01:193
2687133999184,4cyclictest79-21hwrng21:29:522
2687133999177,8cyclictest79-21hwrng21:07:202
2687133999177,8cyclictest79-21hwrng21:07:202
268713199912,81cyclictest2737083-21ntp_states21:16:020
268713199912,81cyclictest2737083-21ntp_states21:16:020
2687135999013,68cyclictest2777775-21apt-get22:15:203
2687133999078,5cyclictest79-21hwrng23:40:562
2687133999078,5cyclictest79-21hwrng23:40:562
2687132998963,21cyclictest301ktimers/122:45:091
2687132998963,21cyclictest301ktimers/122:45:091
2687132998962,15cyclictest0-21swapper/122:21:181
2687132998962,15cyclictest0-21swapper/122:21:181
2687131998983,4cyclictest79-21hwrng20:58:070
2687135998827,28cyclictest2795288-21sshd-auth22:40:273
268713599881,19cyclictest2816342-21ssh23:10:283
2687133998879,7cyclictest0-21swapper/221:21:112
268713199887,75cyclictest2812510-21ssh23:05:100
2687135998729,5cyclictest2826871-21sort23:25:313
2687135998729,5cyclictest2826871-21sort23:25:313
2687135998729,15cyclictest0-21swapper/320:41:083
2687135998729,15cyclictest0-21swapper/320:41:083
268713599871,21cyclictest2695322-21gltestperf19:30:313
2687133998780,5cyclictest79-21hwrng20:37:382
2687131998767,12cyclictest171ktimers/021:40:110
2687131998745,39cyclictest0-21swapper/022:40:580
2687131998714,67cyclictest2816468-21/usr/sbin/munin23:10:310
268713599866,25cyclictest2720125-21ps20:36:003
2687135998635,43cyclictest0-21swapper/320:49:573
2687135998633,10cyclictest0-21swapper/323:39:543
2687135998537,41cyclictest0-21swapper/300:25:083
268713599851,47cyclictest2800265-21users22:46:083
268713599851,47cyclictest2800265-21users22:46:083
2687133998579,4cyclictest79-21hwrng20:18:112
2687132998573,10cyclictest0-21swapper/123:40:501
2687132998573,10cyclictest0-21swapper/123:40:501
2687132998557,23cyclictest301ktimers/122:15:081
2687132998552,24cyclictest0-21swapper/119:40:591
268713299853,57cyclictest2862167-21ssh00:15:421
2687131998546,37cyclictest171ktimers/023:00:590
2687131998546,37cyclictest171ktimers/023:00:590
2687135998467,12cyclictest461ktimers/320:30:103
268713599843,34cyclictest10950irq/205-8000000.ethernet-rx023:06:283
268713599841,45cyclictest2712964-21tail20:16:033
2687133998477,4cyclictest79-21hwrng20:47:532
2687133998476,5cyclictest79-21hwrng20:55:032
2687132998475,5cyclictest10950irq/205-8000000.ethernet-rx020:25:091
2687135998355,22cyclictest0-21swapper/321:36:373
2687133998353,22cyclictest10650irq/155-8000000.ethernet-tx522:39:572
268713199833,60cyclictest16-21ksoftirqd/000:10:360
2687135998246,30cyclictest0-21swapper/320:50:163
268713599821,17cyclictest2747498-21ssh21:30:523
268713599821,17cyclictest2747498-21ssh21:30:523
2687133998268,12cyclictest37-21rcuc/223:55:442
2687133998252,20cyclictest0-21swapper/221:41:162
2687132998276,4cyclictest301ktimers/122:30:491
2687132998258,20cyclictest301ktimers/120:50:311
2687132998248,28cyclictest0-21swapper/121:45:321
2687132998248,28cyclictest0-21swapper/121:45:321
2687132998237,39cyclictest0-21swapper/122:00:401
2687131998251,25cyclictest2711299-21kworker/u16:4+xprtiod20:25:090
2687131998249,27cyclictest0-21swapper/020:21:070
2687135998175,3cyclictest2735908-21apt-get21:15:193
2687135998175,3cyclictest2735908-21apt-get21:15:193
268713299814,12cyclictest2841765-21/usr/sbin/munin23:46:041
268713299814,12cyclictest2841765-21/usr/sbin/munin23:46:041
2687131998160,17cyclictest0-21swapper/021:30:400
2687131998160,17cyclictest0-21swapper/021:30:400
2687131998156,21cyclictest171ktimers/019:45:090
2687135998034,7cyclictest10950irq/205-8000000.ethernet-rx019:39:563
2687135998014,27cyclictest2703700-21users19:51:053
2687133998068,5cyclictest37-21rcuc/222:20:352
2687133998068,5cyclictest37-21rcuc/222:20:352
2687132998058,18cyclictest0-21swapper/120:55:531
2687135997934,9cyclictest0-21swapper/320:56:083
268713599791,19cyclictest2830866-21/usr/sbin/munin23:30:453
2687132997960,11cyclictest0-21swapper/121:00:351
2687132997960,11cyclictest0-21swapper/121:00:351
2687132997935,40cyclictest0-21swapper/100:01:141
268713199793,70cyclictest21-21rcuc/020:44:560
268713199793,67cyclictest2750624-21iwlist21:35:310
2687133997768,6cyclictest79-21hwrng20:23:182
2687132997758,14cyclictest2770820-21apt-get22:05:061
2687132997758,14cyclictest2770820-21apt-get22:05:061
2687132997752,18cyclictest301ktimers/121:25:111
268713299774,35cyclictest2878518-21wc00:39:571
2687131997737,21cyclictest0-21swapper/019:15:090
2687131997735,36cyclictest0-21swapper/019:21:070
2687131997723,12cyclictest10650irq/155-8000000.ethernet-tx520:41:000
2687131997723,12cyclictest10650irq/155-8000000.ethernet-tx520:41:000
2687133997670,4cyclictest79-21hwrng19:30:032
268713199768,65cyclictest2851222-21df_inode00:00:360
2687131997637,37cyclictest0-21swapper/000:29:550
2687131997629,39cyclictest0-21swapper/023:41:390
2687131997629,39cyclictest0-21swapper/023:41:390
2687135997551,19cyclictest0-21swapper/323:55:343
2687131997564,9cyclictest171ktimers/022:05:510
2687131997564,9cyclictest171ktimers/022:05:510
268713199755,41cyclictest16-21ksoftirqd/000:09:350
268713199755,41cyclictest16-21ksoftirqd/000:09:350
268713199752,60cyclictest2862289-21ssh00:15:510
2687135997432,15cyclictest461ktimers/320:05:583
2687133997462,9cyclictest381ktimers/221:30:232
2687133997462,9cyclictest381ktimers/221:30:232
2687133997448,19cyclictest0-21swapper/219:40:352
2687132997446,24cyclictest301ktimers/119:25:101
2687132997446,24cyclictest301ktimers/119:25:101
2687131997426,43cyclictest16-21ksoftirqd/022:55:260
2687132997351,19cyclictest301ktimers/120:10:491
2687132997351,19cyclictest301ktimers/120:10:491
2687132997349,13cyclictest0-21swapper/123:59:561
268713199731,31cyclictest0-21swapper/022:49:280
268713199731,31cyclictest0-21swapper/022:49:280
2687132997263,5cyclictest79-21hwrng20:40:421
2687132997263,5cyclictest79-21hwrng20:40:421
2687132997257,4cyclictest10950irq/205-8000000.ethernet-rx022:36:071
2687132997256,12cyclictest301ktimers/119:45:091
268713299721,55cyclictest31-21ksoftirqd/121:36:331
2687132997166,3cyclictest301ktimers/122:14:271
2687132997162,7cyclictest0-21swapper/123:10:151
2687132997151,18cyclictest301ktimers/120:05:191
2687135997038,26cyclictest0-21swapper/319:09:553
2687135997033,7cyclictest461ktimers/319:55:133
2687133997038,19cyclictest37-21rcuc/220:30:062
2687132997035,3cyclictest2834289-21/usr/sbin/munin23:35:391
2687131997038,6cyclictest10650irq/155-8000000.ethernet-tx522:21:090
2687131997038,6cyclictest10650irq/155-8000000.ethernet-tx522:21:090
2687133996928,34cyclictest0-21swapper/221:35:462
2687132996945,17cyclictest301ktimers/122:26:011
2687132996931,5cyclictest29-21rcuc/123:02:541
2687132996931,5cyclictest29-21rcuc/123:02:541
268713199698,51cyclictest16-21ksoftirqd/021:58:150
268713199698,51cyclictest16-21ksoftirqd/021:58:150
268713199698,51cyclictest16-21ksoftirqd/021:58:150
2687131996955,12cyclictest171ktimers/023:30:110
2687131996948,17cyclictest0-21swapper/022:00:480
2687132996827,36cyclictest2795693-21head22:40:371
2687131996835,29cyclictest2866712-21kworker/u16:13+rpciod00:30:390
2687133996757,7cyclictest381ktimers/223:45:092
2687133996757,7cyclictest381ktimers/223:45:092
2687133996747,17cyclictest0-21swapper/200:16:042
2687133996739,24cyclictest0-21swapper/220:11:062
2687133996739,24cyclictest0-21swapper/220:11:062
268713399672,47cyclictest2816128-21aten_r8power_en23:10:242
2687133996622,25cyclictest2669224-21kworker/u16:13+xprtiod20:50:072
2687132996635,29cyclictest0-21swapper/121:54:561
268713199665,54cyclictest16-21ksoftirqd/023:25:350
268713199665,54cyclictest16-21ksoftirqd/023:25:350
2687131996635,15cyclictest0-21swapper/019:41:130
2687132996553,5cyclictest0-21swapper/123:50:121
2687132996520,28cyclictest0-21swapper/121:55:581
2687132996520,28cyclictest0-21swapper/121:55:581
2687132996520,28cyclictest0-21swapper/121:55:581
2687132996446,13cyclictest301ktimers/122:55:251
2687132996443,17cyclictest0-21swapper/100:07:511
2687132996443,17cyclictest0-21swapper/100:07:511
2687132996443,12cyclictest0-21swapper/121:21:051
2687132996441,20cyclictest0-21swapper/122:54:551
2687131996446,15cyclictest171ktimers/020:50:330
2687131996439,23cyclictest0-21swapper/020:35:420
2687132996338,14cyclictest0-21swapper/100:25:381
2687131996331,28cyclictest2848362-21ssh23:55:530
2687131996330,7cyclictest0-21swapper/023:19:570
2687131996330,7cyclictest0-21swapper/023:19:570
2687131996330,31cyclictest0-21swapper/019:59:560
2687131996330,29cyclictest2785884-21ssh22:26:000
268713199632,55cyclictest2774313-21ssh22:10:090
2687133996230,11cyclictest2685169-21kworker/u16:10+rpciod19:39:572
2687131996239,20cyclictest171ktimers/022:15:180
268713199621,29cyclictest2834348-21ssh23:35:430
268713299616,42cyclictest2735233-21sh21:14:201
2687132996150,9cyclictest301ktimers/123:05:181
2687132996139,14cyclictest0-21swapper/120:31:041
2687132996138,19cyclictest301ktimers/119:10:091
2687132996037,19cyclictest56-21kcompactd023:31:091
268713299601,23cyclictest2732237-21latency_hist21:09:571
268713299601,23cyclictest2732237-21latency_hist21:09:571
2687131996029,21cyclictest171ktimers/021:10:120
2687131995929,9cyclictest0-21swapper/019:10:290
2687133995839,15cyclictest2820597-21ssh23:15:572
2687133995839,15cyclictest2820597-21ssh23:15:572
268713299585,42cyclictest2858530-21memory00:10:351
268713299581,42cyclictest2689238-21apt-get19:15:091
2687132995737,18cyclictest0-21swapper/119:34:551
2687131995734,21cyclictest171ktimers/021:05:080
2687131995734,21cyclictest171ktimers/021:05:080
2687131995733,19cyclictest21-21rcuc/023:45:340
2687131995733,19cyclictest21-21rcuc/023:45:340
2687131995626,26cyclictest2845243-21/usr/sbin/munin23:51:050
2687133995428,16cyclictest2704480-21diskstats19:55:292
2687132995438,12cyclictest29-21rcuc/119:50:251
2687132995237,11cyclictest0-21swapper/120:19:551
2687132995234,16cyclictest301ktimers/119:35:201
2687131995133,15cyclictest0-21swapper/019:09:550
2687133994327,13cyclictest0-21swapper/219:09:552
2687132994126,12cyclictest0-21swapper/119:09:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional