You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 19:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot2.osadl.org (updated Mon Nov 24, 2025 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
256552189174,10sleep10-21swapper/107:09:491
253912183166,12sleep00-21swapper/007:06:210
256302182164,13sleep30-21swapper/307:09:293
23451211194,12sleep20-21swapper/207:05:072
246992420,0sleep00-21swapper/009:27:050
66062120,0sleep10-21swapper/111:59:591
326312120,0sleep30-21swapper/311:07:453
121362110,0sleep10-21swapper/110:41:011
258409982,3cyclictest16321-21perf11:00:010
258469975,1cyclictest31-21ksoftirqd/210:40:132
19043270,0sleep00-21swapper/012:05:190
27220260,0sleep20-21swapper/209:12:262
258509961,4cyclictest0-21swapper/311:50:123
258509960,5cyclictest9030-21apt-get10:40:123
258509955,0cyclictest28427-21kworker/3:010:26:223
258509955,0cyclictest0-21swapper/309:14:053
258509954,1cyclictest24607-21ssh12:08:403
258509950,5cyclictest22622-21memory08:05:173
258509950,5cyclictest22622-21memory08:05:173
258509950,5cyclictest0-21swapper/311:14:063
258509950,4cyclictest1634-21/usr/sbin/munin11:40:303
258509950,4cyclictest0-21swapper/310:00:143
258509950,4cyclictest0-21swapper/309:00:103
258509950,4cyclictest0-21swapper/308:45:133
258469955,0cyclictest0-21swapper/210:22:002
258469952,3cyclictest714-21snmpd11:13:192
258469950,5cyclictest19113-21ssh12:21:532
258469950,5cyclictest0-21swapper/212:37:272
258469950,5cyclictest0-21swapper/210:02:212
258469950,4cyclictest29030-21ssh10:50:112
258469950,0cyclictest0-21swapper/211:51:292
258419951,3cyclictest0-21swapper/111:35:121
258419951,3cyclictest0-21swapper/109:15:121
258419951,2cyclictest0-21swapper/112:35:131
258419951,2cyclictest0-21swapper/111:00:121
258419951,2cyclictest0-21swapper/109:10:131
258409952,3cyclictest13352-21kworker/0:111:45:160
258409951,4cyclictest13352-21kworker/0:111:06:190
258409951,4cyclictest13352-21kworker/0:109:31:160
258409950,5cyclictest13352-21kworker/0:110:05:190
258409950,5cyclictest0-21swapper/007:15:210
258409950,0cyclictest13352-21kworker/0:111:25:410
258409950,0cyclictest0-21swapper/009:24:520
258509944,0cyclictest28427-21kworker/3:009:08:013
258509944,0cyclictest28427-21kworker/3:008:56:483
258509944,0cyclictest0-21swapper/312:36:533
258509942,2cyclictest14622-21ssh09:54:453
258509941,3cyclictest55950irq/130-eth012:20:123
258509941,3cyclictest4224-21sh09:16:583
258509940,4cyclictest9778-21apt-get11:45:133
258509940,4cyclictest27362-21ssh12:25:423
258509940,4cyclictest25483-21ssh10:15:223
258509940,4cyclictest20438-21df_inode09:25:153
258509940,4cyclictest0-21swapper/312:16:113
258509940,4cyclictest0-21swapper/312:12:523
258509940,4cyclictest0-21swapper/311:30:113
258509940,4cyclictest0-21swapper/311:25:243
258509940,4cyclictest0-21swapper/311:20:123
258509940,4cyclictest0-21swapper/311:15:533
258509940,4cyclictest0-21swapper/310:50:113
258509940,4cyclictest0-21swapper/310:36:173
258509940,4cyclictest0-21swapper/310:22:223
258509940,4cyclictest0-21swapper/310:10:143
258509940,4cyclictest0-21swapper/308:20:143
258509940,4cyclictest0-21swapper/307:20:113
258509940,4cyclictest0-21swapper/307:17:393
258509940,3cyclictest9120-21sort07:40:003
258509940,3cyclictest8172-21df09:35:163
258509940,3cyclictest6983-21cat12:00:003
258509940,3cyclictest6559-21apt-get10:55:023
258509940,3cyclictest3736-21cat10:05:153
258509940,3cyclictest30472-21df_inode09:30:123
258509940,3cyclictest27744-21ssh10:49:313
258509940,3cyclictest2611-21sh11:57:273
258509940,3cyclictest19898-21rm09:56:463
258509940,3cyclictest19586-21df_inode08:00:123
258509940,3cyclictest17727-21cut09:40:143
258509940,3cyclictest16617-21/usr/sbin/munin07:55:123
258509940,1cyclictest6656-21/usr/sbin/munin08:40:183
258509940,1cyclictest2987-21df_inode12:30:123
258509940,1cyclictest24989-21irqstats08:10:143
258509940,1cyclictest22393-21expr11:35:123
258509940,1cyclictest21327-21/usr/sbin/munin10:30:223
258509940,0cyclictest0-21swapper/311:00:353
258509940,0cyclictest0-21swapper/309:47:583
258509940,0cyclictest0-21swapper/309:20:173
258469944,0cyclictest0-21swapper/211:35:022
258469944,0cyclictest0-21swapper/211:05:482
258469944,0cyclictest0-21swapper/208:17:082
258469944,0cyclictest0-21swapper/207:35:242
258469943,1cyclictest31-21ksoftirqd/208:50:002
258469943,1cyclictest26601-21sort07:10:142
258469940,4cyclictest6879-21ssh09:50:182
258469940,4cyclictest19919-21ssh12:05:412
258469940,4cyclictest11057-21ssh10:08:552
258469940,4cyclictest0-21swapper/212:25:132
258469940,4cyclictest0-21swapper/212:10:132
258469940,4cyclictest0-21swapper/212:00:012
258469940,4cyclictest0-21swapper/211:55:132
258469940,4cyclictest0-21swapper/211:41:342
258469940,4cyclictest0-21swapper/211:25:122
258469940,4cyclictest0-21swapper/211:20:472
258469940,4cyclictest0-21swapper/210:35:172
258469940,4cyclictest0-21swapper/210:30:122
258469940,4cyclictest0-21swapper/210:15:032
258469940,4cyclictest0-21swapper/210:10:212
258469940,4cyclictest0-21swapper/209:55:122
258469940,4cyclictest0-21swapper/209:35:132
258469940,4cyclictest0-21swapper/209:25:132
258469940,4cyclictest0-21swapper/208:54:152
258469940,3cyclictest9513-21ssh10:55:442
258469940,3cyclictest3348-21rm09:16:262
258469940,3cyclictest30818-21users08:20:222
258469940,3cyclictest29974-21ssh09:45:252
258469940,3cyclictest28434-21diskstats07:15:002
258469940,3cyclictest26249-21latency_hist08:15:002
258469940,3cyclictest2424-21rm09:32:072
258469940,3cyclictest21460-21cat09:10:002
258469940,3cyclictest21166-21ssh11:18:132
258469940,3cyclictest19780-21tune2fs10:45:152
258469940,3cyclictest18004-21cat11:00:172
258469940,3cyclictest17812-21ssh11:49:092
258469940,3cyclictest17706-21/usr/sbin/munin09:40:142
258469940,3cyclictest12776-21ssh09:20:582
258469940,3cyclictest12477-21rm11:30:132
258469940,3cyclictest10441-21ssh12:17:432
258419944,0cyclictest0-21swapper/110:19:161
258419942,1cyclictest0-21swapper/110:10:141
258419941,2cyclictest0-21swapper/112:25:001
258419941,2cyclictest0-21swapper/112:20:111
258419941,2cyclictest0-21swapper/112:00:001
258419941,2cyclictest0-21swapper/111:25:011
258419941,2cyclictest0-21swapper/110:55:111
258419941,2cyclictest0-21swapper/110:45:131
258419941,2cyclictest0-21swapper/110:25:021
258419941,2cyclictest0-21swapper/110:20:121
258419941,2cyclictest0-21swapper/109:35:131
258419941,2cyclictest0-21swapper/109:00:111
258419941,2cyclictest0-21swapper/108:35:111
258419941,2cyclictest0-21swapper/108:05:141
258419941,2cyclictest0-21swapper/108:05:141
258419941,2cyclictest0-21swapper/107:25:121
258419941,2cyclictest0-21swapper/107:20:101
258419941,2cyclictest0-21swapper/107:15:131
258419941,1cyclictest0-21swapper/112:15:101
258419940,4cyclictest0-21swapper/111:15:261
258419940,4cyclictest0-21swapper/111:07:071
258419940,4cyclictest0-21swapper/109:55:421
258419940,3cyclictest0-21swapper/111:30:111
258419940,2cyclictest0-21swapper/110:50:121
258419940,2cyclictest0-21swapper/110:30:131
258419940,2cyclictest0-21swapper/108:30:121
258419940,0cyclictest0-21swapper/109:48:501
258409944,0cyclictest0-21swapper/012:30:220
258409944,0cyclictest0-21swapper/011:13:330
258409944,0cyclictest0-21swapper/011:00:010
258409944,0cyclictest0-21swapper/010:32:500
258409943,1cyclictest21881-21tr08:05:140
258409943,1cyclictest21881-21tr08:05:140
258409943,1cyclictest16103-21ssh12:20:170
258409942,2cyclictest13352-21kworker/0:111:30:310
258409941,3cyclictest714-21snmpd11:21:560
258409940,4cyclictest32206-21ssh10:03:100
258409940,4cyclictest21272-21diskmemload12:14:290
258409940,4cyclictest0-21swapper/010:53:490
258409940,4cyclictest0-21swapper/010:25:130
258409940,4cyclictest0-21swapper/010:15:090
258409940,4cyclictest0-21swapper/009:45:140
258409940,4cyclictest0-21swapper/009:41:190
258409940,4cyclictest0-21swapper/009:10:140
258409940,4cyclictest0-21swapper/007:10:120
258409940,3cyclictest9695-21diskstats07:40:130
258409940,3cyclictest9452-21date11:45:020
258409940,3cyclictest8870-21grep12:00:160
258409940,3cyclictest6781-21dpkg12:00:000
258409940,3cyclictest31957-21df_inode10:35:140
258409940,3cyclictest27146-21perl08:15:150
258409940,3cyclictest23013-21cut10:15:000
258409940,3cyclictest22963-21sh11:51:060
258409940,3cyclictest19024-21/usr/sbin/munin09:05:160
258409940,3cyclictest1849-21sh09:15:300
258409940,3cyclictest1723-21ssh10:20:140
258409940,3cyclictest15378-21ntp_states07:50:180
258409940,3cyclictest15293-21perl11:15:130
258409940,3cyclictest12069-21ssh12:35:010
258409940,2cyclictest0-21swapper/008:25:120
258409940,0cyclictest0-21swapper/012:16:170
19670240,0sleep10-21swapper/109:25:081
5346230,0sleep10-21swapper/109:50:101
258509930,3cyclictest9620-21cat07:40:123
258509930,3cyclictest4727-21df07:30:143
258509930,3cyclictest31439-21/usr/sbin/munin08:25:133
258509930,3cyclictest12115-21df_inode08:50:103
258509930,3cyclictest0-21swapper/308:35:133
258509930,3cyclictest0-21swapper/308:15:023
258509930,3cyclictest0-21swapper/307:50:123
258509930,3cyclictest0-21swapper/307:45:113
258509930,3cyclictest0-21swapper/307:25:023
258509930,3cyclictest0-21swapper/307:10:063
258509930,0cyclictest0-21swapper/308:30:123
258469933,0cyclictest7758-21kworker/2:107:55:102
258469933,0cyclictest7758-21kworker/2:107:45:012
258469933,0cyclictest7758-21kworker/2:107:30:062
258469932,1cyclictest714-21snmpd07:50:122
258469931,1cyclictest4828-21tune2fs08:35:142
258469931,1cyclictest2245-21ssh12:30:042
258469930,3cyclictest0-21swapper/210:25:012
258469930,3cyclictest0-21swapper/209:00:072
258469930,3cyclictest0-21swapper/208:55:092
258469930,3cyclictest0-21swapper/208:40:042
258469930,3cyclictest0-21swapper/208:30:012
258469930,3cyclictest0-21swapper/208:25:082
258469930,3cyclictest0-21swapper/208:05:122
258469930,3cyclictest0-21swapper/208:05:122
258469930,3cyclictest0-21swapper/208:00:122
258469930,3cyclictest0-21swapper/207:40:112
258469930,3cyclictest0-21swapper/207:25:122
258469930,3cyclictest0-21swapper/207:20:102
258419931,2cyclictest0-21swapper/112:30:011
258419931,1cyclictest0-21swapper/112:05:121
258419931,1cyclictest0-21swapper/111:45:121
258419931,1cyclictest0-21swapper/111:40:131
258419931,1cyclictest0-21swapper/111:20:111
258419931,1cyclictest0-21swapper/110:35:121
258419931,1cyclictest0-21swapper/109:40:131
258419931,1cyclictest0-21swapper/109:05:131
258419931,1cyclictest0-21swapper/108:50:091
258419931,1cyclictest0-21swapper/108:20:121
258419931,1cyclictest0-21swapper/107:55:121
258419931,1cyclictest0-21swapper/107:45:121
258419931,1cyclictest0-21swapper/107:40:121
258419930,3cyclictest0-21swapper/108:58:301
258419930,3cyclictest0-21swapper/108:04:121
258419930,2cyclictest29503-21taskset12:10:291
258419930,2cyclictest0-21swapper/111:10:141
258419930,2cyclictest0-21swapper/110:00:131
258419930,2cyclictest0-21swapper/109:30:121
258419930,2cyclictest0-21swapper/108:10:101
258409932,1cyclictest714-21snmpd09:01:000
258409932,1cyclictest714-21snmpd07:35:300
258409931,1cyclictest0-21swapper/007:55:120
258409930,3cyclictest7634-21apt-get09:35:140
258409930,3cyclictest28975-21apt-get08:20:120
258409930,3cyclictest24290-21dpkg12:25:000
258409930,3cyclictest24214-21cpuspeed08:10:120
258409930,3cyclictest22693-21df11:35:120
258409930,3cyclictest21272-21diskmemload10:40:040
258409930,3cyclictest0-21swapper/010:45:070
258409930,3cyclictest0-21swapper/009:55:060
258409930,3cyclictest0-21swapper/009:50:100
258409930,3cyclictest0-21swapper/008:55:100
258409930,3cyclictest0-21swapper/008:50:090
258409930,3cyclictest0-21swapper/008:45:120
258409930,3cyclictest0-21swapper/008:40:130
258409930,3cyclictest0-21swapper/008:35:120
258409930,3cyclictest0-21swapper/008:30:110
258409930,3cyclictest0-21swapper/008:00:110
258409930,3cyclictest0-21swapper/007:45:120
258409930,3cyclictest0-21swapper/007:30:110
258409930,3cyclictest0-21swapper/007:25:150
258409930,3cyclictest0-21swapper/007:20:070
258419920,1cyclictest259572sleep107:10:131
258419920,1cyclictest0-21swapper/111:50:001
258419920,1cyclictest0-21swapper/110:05:131
258419920,1cyclictest0-21swapper/109:20:121
258419920,1cyclictest0-21swapper/108:45:121
258419920,1cyclictest0-21swapper/108:40:131
258419920,1cyclictest0-21swapper/108:25:131
258419920,1cyclictest0-21swapper/108:15:111
258419920,1cyclictest0-21swapper/107:50:111
258419920,1cyclictest0-21swapper/107:35:111
258419920,1cyclictest0-21swapper/107:30:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional