You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 23:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Sat Nov 22, 2025 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25049991918,1cyclictest66850irq/125-lan11:45:203
25049991918,1cyclictest66850irq/125-lan11:41:193
25049991918,1cyclictest66850irq/125-lan10:54:273
25049991918,1cyclictest66850irq/125-lan09:45:233
25049991918,1cyclictest66850irq/125-lan09:10:123
25049991918,1cyclictest66850irq/125-lan09:05:293
25049991818,0cyclictest66850irq/125-lan12:35:563
25049991818,0cyclictest66850irq/125-lan12:30:233
25049991818,0cyclictest66850irq/125-lan12:18:233
25049991818,0cyclictest66850irq/125-lan12:10:103
25049991818,0cyclictest66850irq/125-lan11:39:243
25049991818,0cyclictest66850irq/125-lan11:02:073
25049991817,1cyclictest66850irq/125-lan10:25:173
25049991817,1cyclictest66850irq/125-lan10:20:203
25049991817,1cyclictest66850irq/125-lan09:55:193
25049991817,1cyclictest66850irq/125-lan09:35:523
25049991817,1cyclictest66850irq/125-lan08:10:423
25049991817,1cyclictest66850irq/125-lan07:27:013
25049991717,0cyclictest66850irq/125-lan12:25:163
25049991717,0cyclictest66850irq/125-lan11:58:293
25049991717,0cyclictest66850irq/125-lan11:22:553
25049991717,0cyclictest66850irq/125-lan11:13:083
25049991717,0cyclictest66850irq/125-lan11:06:343
25049991717,0cyclictest66850irq/125-lan09:54:063
25049991717,0cyclictest66850irq/125-lan09:21:013
25049991717,0cyclictest66850irq/125-lan07:32:063
25049991717,0cyclictest66850irq/125-lan07:10:223
25049991716,1cyclictest66850irq/125-lan12:20:183
25049991716,1cyclictest66850irq/125-lan12:07:173
25049991716,1cyclictest66850irq/125-lan11:50:203
25049991716,1cyclictest66850irq/125-lan11:25:163
25049991716,1cyclictest66850irq/125-lan11:17:273
25049991716,1cyclictest66850irq/125-lan10:56:223
25049991716,1cyclictest66850irq/125-lan10:45:183
25049991716,1cyclictest66850irq/125-lan10:44:593
25049991716,1cyclictest66850irq/125-lan10:36:523
25049991716,1cyclictest66850irq/125-lan10:17:233
25049991716,1cyclictest66850irq/125-lan10:10:533
25049991716,1cyclictest66850irq/125-lan10:09:163
25049991716,1cyclictest66850irq/125-lan10:03:523
25049991716,1cyclictest66850irq/125-lan09:40:213
25049991716,1cyclictest66850irq/125-lan09:31:503
25049991716,1cyclictest66850irq/125-lan09:29:043
25049991716,1cyclictest66850irq/125-lan09:17:143
25049991716,1cyclictest66850irq/125-lan07:55:153
25049991716,1cyclictest66850irq/125-lan07:50:173
25049991716,1cyclictest66850irq/125-lan07:40:223
25049991716,1cyclictest66850irq/125-lan07:35:223
25049991616,0cyclictest66850irq/125-lan11:30:573
25049991615,1cyclictest66850irq/125-lan12:01:593
25049991615,1cyclictest66850irq/125-lan10:32:213
25049991615,1cyclictest66850irq/125-lan08:05:193
25049991615,1cyclictest66850irq/125-lan08:00:223
25049991614,1cyclictest66850irq/125-lan07:20:173
25049991515,0cyclictest66850irq/125-lan08:43:443
25049991514,1cyclictest66850irq/125-lan08:30:143
25049991414,0cyclictest66850irq/125-lan08:25:163
25049991413,1cyclictest66850irq/125-lan08:55:113
25049991413,1cyclictest66850irq/125-lan07:45:263
25049991313,0cyclictest66850irq/125-lan07:15:233
25049991312,1cyclictest66850irq/125-lan08:15:153
25049991211,1cyclictest66850irq/125-lan08:35:223
25049991111,0cyclictest66850irq/125-lan08:45:463
2504399112,6cyclictest12206-21chrt10:25:562
25043991110,1cyclictest517-21systemd-resolve12:30:202
2504999109,1cyclictest66850irq/125-lan08:50:173
2504999109,1cyclictest66850irq/125-lan08:20:153
2504399102,6cyclictest9942-21diskmemload09:40:382
2504399102,6cyclictest31973-21seq08:41:352
2504399102,6cyclictest20803-21runrttasks11:49:042
2504399102,6cyclictest17864-21seq11:09:432
2504399102,5cyclictest9942-21diskmemload11:30:402
2504399102,5cyclictest9678-21ssh11:00:062
2504399102,5cyclictest5833-21ssh10:55:142
2504399102,5cyclictest32442-21chrt10:12:232
2504399102,5cyclictest25555-21sh11:17:472
2503999102,6cyclictest18448-21ssh09:56:211
250499998,1cyclictest66850irq/125-lan09:00:183
250439998,1cyclictest771-21snmpd08:54:102
250439998,1cyclictest524-21avahi-daemon11:50:312
250439996,3cyclictest31629-21cat12:35:162
250439993,4cyclictest2468-21chrt07:32:452
250439993,3cyclictest15200-21ssh11:41:442
250439992,5cyclictest9573-21taskset09:05:292
250439992,5cyclictest26863-21seq07:13:452
250439992,5cyclictest25706-21ssh10:41:032
250439992,5cyclictest24593-21ssh10:03:572
250439992,5cyclictest24539-21ssh12:28:012
250439992,5cyclictest1725-21chrt08:48:182
250439992,5cyclictest12714-21ntpq07:55:182
250439992,4cyclictest8616-21taskset07:48:082
250439992,4cyclictest583-21chrt07:29:562
250439992,4cyclictest26218-21taskset09:27:222
250439992,4cyclictest24749-21ntpq08:25:172
250439992,4cyclictest21488-21ssh09:21:152
250439992,4cyclictest2017-21sh09:37:332
250439992,4cyclictest19858-21ssh11:10:282
250439992,4cyclictest13799-21ssh12:16:542
250439992,4cyclictest13297-21taskset09:51:102
250439992,4cyclictest10487-21ssh11:36:282
250439992,3cyclictest31063-21ssh10:47:382
250439991,6cyclictest9942-21diskmemload10:17:432
250439991,6cyclictest29919-21taskset08:36:182
250439991,6cyclictest29040-21sendmail-msp07:20:012
250439991,6cyclictest1853-21ssh11:27:022
250439991,6cyclictest11878-21sh09:10:292
250439991,5cyclictest5569-21seq08:55:562
250399992,5cyclictest27028-21ssh10:43:191
250399991,6cyclictest9942-21diskmemload09:44:051
250399991,6cyclictest9942-21diskmemload09:23:141
250399991,6cyclictest32020-21ntpq12:35:191
250399991,6cyclictest11920-21ssh11:38:581
250399991,5cyclictest3688-21taskset08:52:441
250399991,5cyclictest31922-21fw_conntrack09:35:171
250399991,4cyclictest4214-21ssh12:06:191
250399991,4cyclictest18800-21basename11:10:141
250329998,1cyclictest769-21lldpd08:45:260
250329997,2cyclictest771-21snmpd09:36:520
250439988,0cyclictest769-21lldpd09:01:362
250439983,3cyclictest7582-21ssh12:10:212
250439982,4cyclictest7983-21ssh10:21:112
250439982,4cyclictest4147-21ssh12:06:142
250439982,4cyclictest28323-21memory09:30:162
250439982,4cyclictest27921-21chrt08:31:332
250439982,4cyclictest19552-21chrt08:13:192
250439982,3cyclictest26428-21ntpq10:05:182
250439982,3cyclictest21351-21taskset08:15:382
250439982,3cyclictest20882-21ssh10:35:512
250439981,6cyclictest9942-21diskmemload09:57:042
250439981,5cyclictest9942-21diskmemload11:21:442
250439981,5cyclictest9942-21diskmemload10:33:542
250439981,5cyclictest7864-21ssh09:45:072
250439981,5cyclictest30151-21ntpq07:20:212
250439981,5cyclictest2382-21ssh10:50:342
250439981,5cyclictest10255-21ntpq07:50:212
250439981,4cyclictest9942-21diskmemload12:20:032
250439981,4cyclictest2899-21seq07:35:062
250439981,4cyclictest0-21swapper/208:05:522
250439981,3cyclictest30383-21ssh12:00:102
250439981,3cyclictest27529-21ssh11:55:232
250399987,1cyclictest524-21avahi-daemon10:48:581
250399986,2cyclictest771-21snmpd09:25:381
250399986,2cyclictest749-21ntpd08:59:451
250399986,2cyclictest749-21ntpd08:35:481
250399986,2cyclictest749-21ntpd07:35:071
250399986,2cyclictest749-21ntpd07:28:221
250399982,5cyclictest7859-21ssh10:20:581
250399982,4cyclictest7904-21ntpq07:45:231
250399982,4cyclictest30924-21chrt07:23:561
250399982,4cyclictest27849-21seq08:30:391
250399982,3cyclictest19023-21sensors_temp08:10:421
250399982,3cyclictest12329-21ssh09:11:121
250399981,6cyclictest11221-21sh09:49:321
250399981,5cyclictest9942-21diskmemload12:01:161
250399981,5cyclictest9942-21diskmemload11:45:061
250399981,5cyclictest9942-21diskmemload11:06:061
250399981,5cyclictest5986-21ssh11:31:311
250399981,5cyclictest5806-21ntpq07:40:181
250399981,5cyclictest31205-21ntp_192.168.11408:40:171
250399981,5cyclictest30197-21taskset09:32:491
250399981,5cyclictest2753-21ntp_states10:15:201
250399981,5cyclictest27479-21irqstats11:20:151
250399981,5cyclictest26945-21ntpq11:55:181
250399981,5cyclictest21955-21ntpq10:00:181
250399981,5cyclictest1671-21ntpq07:30:181
250399981,5cyclictest11537-21ssh11:00:561
250399981,5cyclictest10177-21ntp_192.168.11407:50:211
250399981,5cyclictest0-21swapper/108:22:231
250399981,4cyclictest9942-21diskmemload11:15:071
250399981,4cyclictest9942-21diskmemload10:29:171
250399981,4cyclictest6296-21ssh12:10:011
250399981,4cyclictest550-21NetworkManager10:10:081
250399981,4cyclictest27950-21ntpq07:15:171
250399981,4cyclictest21170-21ssh12:25:001
250399981,4cyclictest16946-21taskset10:32:011
250399981,3cyclictest7754-21seq09:02:521
250399981,3cyclictest26447-21ntpq10:05:191
250399981,3cyclictest15293-21ssh11:41:531
250399980,6cyclictest16224-21ntpq09:15:201
250399980,3cyclictest6265-21ssh10:55:171
250329988,0cyclictest771-21snmpd10:47:060
250329988,0cyclictest771-21snmpd10:38:240
250329988,0cyclictest771-21snmpd08:07:270
250329988,0cyclictest769-21lldpd12:38:390
250329988,0cyclictest769-21lldpd11:36:240
250329988,0cyclictest769-21lldpd09:16:190
250329988,0cyclictest769-21lldpd08:20:010
250329988,0cyclictest769-21lldpd07:26:570
250329987,1cyclictest771-21snmpd12:18:370
250329987,1cyclictest771-21snmpd08:42:230
250329987,1cyclictest769-21lldpd10:24:050
250329987,1cyclictest524-21avahi-daemon12:08:070
250329987,1cyclictest524-21avahi-daemon07:20:190
250329987,1cyclictest255-20systemd-journal09:57:580
250329986,2cyclictest771-21snmpd12:23:190
250329986,2cyclictest771-21snmpd11:49:110
250329986,2cyclictest771-21snmpd07:58:490
250329986,2cyclictest771-21snmpd07:30:330
250329986,2cyclictest771-21snmpd07:10:150
250329986,2cyclictest5454-21cat07:40:160
250439972,3cyclictest14327-21hddtemp_smartct08:00:152
250439970,3cyclictest5444-21cat07:40:152
250439970,3cyclictest0-21swapper/209:15:012
250399977,0cyclictest771-21snmpd10:50:491
250399976,1cyclictest771-21snmpd09:51:131
250399976,1cyclictest769-21lldpd08:25:181
250399975,2cyclictest771-21snmpd07:11:031
250399975,2cyclictest749-21ntpd08:18:551
250399972,3cyclictest839-21ntpq08:45:171
250399972,3cyclictest16786-21ntpq08:05:201
250399972,3cyclictest12751-21ntpq07:55:181
250399972,3cyclictest12126-21ntpq12:15:181
250399972,2cyclictest951-21ssh11:25:311
250399971,5cyclictest9808-21runrttasks09:08:281
250399971,4cyclictest14651-21ntpq08:00:171
250399971,3cyclictest9942-21diskmemload12:30:091
250399971,3cyclictest20507-21unixbench_singl10:35:231
250399971,2cyclictest22071-21grep11:50:161
250399971,2cyclictest16877-21memory12:20:161
250329977,0cyclictest771-21snmpd12:33:450
250329977,0cyclictest771-21snmpd11:05:550
250329977,0cyclictest771-21snmpd10:30:030
250329977,0cyclictest771-21snmpd10:26:320
250329977,0cyclictest771-21snmpd10:05:290
250329977,0cyclictest769-21lldpd11:53:120
250329977,0cyclictest769-21lldpd09:13:550
250329977,0cyclictest769-21lldpd09:01:370
250329976,1cyclictest771-21snmpd08:55:090
250329976,1cyclictest584-21in:imuxsock08:02:130
250329976,1cyclictest524-21avahi-daemon07:45:520
250329975,2cyclictest771-21snmpd12:11:440
250329975,2cyclictest771-21snmpd10:53:580
250329975,2cyclictest771-21snmpd10:16:230
250329975,2cyclictest771-21snmpd08:35:120
250329975,2cyclictest771-21snmpd08:30:530
250329975,2cyclictest21725-21idleruntime-cro08:19:590
250329972,5cyclictest550-21NetworkManager09:08:030
250329965,1cyclictest771-21snmpd10:58:330
250329965,1cyclictest771-21snmpd10:01:260
250329965,1cyclictest32115-21fschecks_time11:25:130
250329965,1cyclictest24150-21gltestperf10:40:140
250329964,2cyclictest771-21snmpd12:00:510
250329964,2cyclictest771-21snmpd11:57:170
250329964,2cyclictest550-21NetworkManager11:21:570
250329964,2cyclictest18255-21tr08:10:140
250329963,2cyclictest25279-21grep09:25:320
250329963,2cyclictest21382-21ssh09:21:040
250329962,3cyclictest4401-21irqstats11:30:140
250329955,0cyclictest771-21snmpd08:51:280
250329955,0cyclictest769-21lldpd07:16:410
250329953,2cyclictest550-21NetworkManager11:17:090
250329952,3cyclictest550-21NetworkManager11:10:370
250329952,2cyclictest8617-21sed09:45:190
250329952,2cyclictest4412-21update-cracklib07:36:450
250329952,2cyclictest3520-21ssh09:40:100
250329952,2cyclictest29971-21dpkg10:10:130
250329952,2cyclictest29962-21ssh09:32:230
250329952,2cyclictest13932-21ssh09:52:150
250329952,2cyclictest11435-21sshd11:00:450
250329951,3cyclictest9476-21apt-get07:50:170
250329951,3cyclictest22711-21sed12:25:210
250329942,2cyclictest13098-21uniq11:40:130
250329940,3cyclictest24306-21fw_conntrack08:25:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional