You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-23 - 07:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot6s.osadl.org (updated Wed Apr 23, 2025 00:43:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21401991919,0cyclictest65850irq/126-lan22:57:152
21401991918,1cyclictest65850irq/126-lan23:37:542
21401991918,1cyclictest65850irq/126-lan21:30:152
21401991918,1cyclictest65850irq/126-lan21:25:292
21401991818,0cyclictest65850irq/126-lan23:10:062
21401991818,0cyclictest65850irq/126-lan22:33:312
21401991818,0cyclictest65850irq/126-lan19:20:192
21401991818,0cyclictest65850irq/126-lan00:06:572
21401991817,1cyclictest65850irq/126-lan23:32:272
21401991817,1cyclictest65850irq/126-lan22:48:132
21401991817,1cyclictest65850irq/126-lan22:35:262
21401991817,1cyclictest65850irq/126-lan22:21:502
21401991817,1cyclictest65850irq/126-lan22:10:002
21401991817,1cyclictest65850irq/126-lan22:08:002
21401991817,1cyclictest65850irq/126-lan21:55:292
21401991817,1cyclictest65850irq/126-lan21:50:422
21401991817,1cyclictest65850irq/126-lan21:40:112
21401991817,1cyclictest65850irq/126-lan21:36:302
21401991817,1cyclictest65850irq/126-lan00:18:162
21401991717,0cyclictest65850irq/126-lan23:05:092
21401991717,0cyclictest65850irq/126-lan21:48:012
21401991717,0cyclictest65850irq/126-lan21:17:492
21401991717,0cyclictest65850irq/126-lan19:55:132
21401991717,0cyclictest65850irq/126-lan00:13:142
21401991716,1cyclictest65850irq/126-lan23:54:362
21401991716,1cyclictest65850irq/126-lan23:40:192
21401991716,1cyclictest65850irq/126-lan23:25:172
21401991716,1cyclictest65850irq/126-lan23:20:282
21401991716,1cyclictest65850irq/126-lan23:00:142
21401991716,1cyclictest65850irq/126-lan22:42:292
21401991716,1cyclictest65850irq/126-lan22:15:172
21401991716,1cyclictest65850irq/126-lan22:00:102
21401991716,1cyclictest65850irq/126-lan21:20:092
21401991716,1cyclictest65850irq/126-lan21:10:182
21401991716,1cyclictest65850irq/126-lan20:40:222
21401991716,1cyclictest65850irq/126-lan20:25:172
21401991716,1cyclictest65850irq/126-lan20:05:142
21401991716,1cyclictest65850irq/126-lan00:21:112
21401991716,1cyclictest65850irq/126-lan00:00:012
21401991616,0cyclictest65850irq/126-lan20:45:182
21401991616,0cyclictest65850irq/126-lan19:50:332
21401991615,1cyclictest65850irq/126-lan23:55:312
21401991615,1cyclictest65850irq/126-lan23:47:432
21401991615,1cyclictest65850irq/126-lan23:15:122
21401991615,1cyclictest65850irq/126-lan22:50:142
21401991615,1cyclictest65850irq/126-lan22:28:192
21401991615,1cyclictest65850irq/126-lan20:23:152
21401991615,1cyclictest65850irq/126-lan20:15:222
21401991615,1cyclictest65850irq/126-lan19:40:162
21401991615,1cyclictest65850irq/126-lan00:36:472
21401991615,1cyclictest65850irq/126-lan00:31:252
21401991515,0cyclictest65850irq/126-lan19:25:422
21401991514,1cyclictest65850irq/126-lan00:26:082
21401991414,0cyclictest65850irq/126-lan20:39:322
21401991414,0cyclictest65850irq/126-lan19:46:232
21401991414,0cyclictest65850irq/126-lan19:35:232
21401991414,0cyclictest65850irq/126-lan19:15:192
21401991413,1cyclictest65850irq/126-lan21:06:332
21401991413,1cyclictest65850irq/126-lan20:55:202
21401991313,0cyclictest65850irq/126-lan20:00:162
21401991313,0cyclictest65850irq/126-lan19:10:162
21401991212,0cyclictest65850irq/126-lan20:52:322
21401991211,1cyclictest65850irq/126-lan20:10:192
21401991110,1cyclictest65850irq/126-lan20:30:142
21401991110,1cyclictest65850irq/126-lan19:30:132
2140599109,1cyclictest765-21snmpd20:31:173
21401991010,0cyclictest65850irq/126-lan21:00:162
2139299109,1cyclictest506-21avahi-daemon23:09:000
214059998,1cyclictest765-21snmpd19:53:183
214059998,1cyclictest763-21lldpd19:48:123
213969998,1cyclictest765-21snmpd21:25:181
213969998,1cyclictest765-21snmpd20:13:491
213969998,1cyclictest765-21snmpd19:44:001
213929998,1cyclictest765-21snmpd21:17:450
213929998,1cyclictest765-21snmpd20:28:410
213929998,1cyclictest506-21avahi-daemon22:21:290
214059988,0cyclictest765-21snmpd21:21:033
214059988,0cyclictest765-21snmpd21:07:573
214059988,0cyclictest765-21snmpd20:46:113
214059988,0cyclictest763-21lldpd23:49:523
214059988,0cyclictest763-21lldpd23:43:423
214059988,0cyclictest763-21lldpd22:20:163
214059988,0cyclictest763-21lldpd20:38:473
214059988,0cyclictest763-21lldpd20:11:093
214059988,0cyclictest763-21lldpd19:56:123
214059987,1cyclictest765-21snmpd21:59:073
214059987,1cyclictest763-21lldpd21:03:033
214059987,1cyclictest763-21lldpd20:18:313
214059987,1cyclictest763-21lldpd19:37:263
214059987,1cyclictest6798-21cut00:00:013
214059987,1cyclictest506-21avahi-daemon20:05:193
214059987,1cyclictest499-21systemd-resolve23:09:123
213969988,0cyclictest765-21snmpd19:14:071
213969988,0cyclictest763-21lldpd21:46:421
213969988,0cyclictest763-21lldpd20:41:411
213969988,0cyclictest763-21lldpd00:02:401
213969987,1cyclictest765-21snmpd23:19:161
213969987,1cyclictest763-21lldpd22:29:131
213969987,1cyclictest763-21lldpd20:18:391
213969987,1cyclictest506-21avahi-daemon23:56:251
213969987,1cyclictest506-21avahi-daemon22:40:061
213969987,1cyclictest506-21avahi-daemon21:16:281
213969987,0cyclictest765-21snmpd22:00:011
213969986,2cyclictest765-21snmpd19:30:121
213929988,0cyclictest765-21snmpd22:28:020
213929988,0cyclictest765-21snmpd00:11:530
213929988,0cyclictest763-21lldpd23:45:100
213929988,0cyclictest763-21lldpd20:55:090
213929987,1cyclictest506-21avahi-daemon23:57:570
213929987,1cyclictest506-21avahi-daemon22:56:580
213929986,2cyclictest765-21snmpd21:41:040
213929986,2cyclictest765-21snmpd21:12:030
213929986,2cyclictest765-21snmpd20:45:440
213929986,2cyclictest765-21snmpd19:27:480
213929986,2cyclictest765-21snmpd00:18:230
213929986,2cyclictest739-21ntpd23:35:080
213929986,2cyclictest739-21ntpd23:16:040
213929986,2cyclictest739-21ntpd23:14:230
213929986,2cyclictest739-21ntpd22:38:590
213929986,2cyclictest739-21ntpd22:01:390
213929986,2cyclictest739-21ntpd21:35:400
213929986,2cyclictest739-21ntpd00:36:510
213929986,2cyclictest739-21ntpd00:31:590
213929986,2cyclictest739-21ntpd00:29:050
213929986,2cyclictest739-21ntpd00:01:090
214059977,0cyclictest765-21snmpd22:12:533
214059977,0cyclictest765-21snmpd21:33:243
214059977,0cyclictest763-21lldpd23:20:533
214059977,0cyclictest763-21lldpd22:27:223
214059977,0cyclictest763-21lldpd21:49:103
214059977,0cyclictest763-21lldpd00:37:303
214059977,0cyclictest763-21lldpd00:23:163
214059977,0cyclictest763-21lldpd00:15:263
214059976,1cyclictest765-21snmpd23:27:493
214059976,1cyclictest765-21snmpd22:56:063
214059976,1cyclictest765-21snmpd20:52:323
214059976,1cyclictest765-21snmpd20:43:143
214059976,1cyclictest765-21snmpd19:42:513
214059976,1cyclictest763-21lldpd19:21:223
214059976,1cyclictest506-21avahi-daemon00:13:463
214059976,1cyclictest25342-21idleruntime-cro21:00:003
214059975,2cyclictest765-21snmpd23:52:383
214059975,2cyclictest765-21snmpd22:34:563
214059975,2cyclictest765-21snmpd22:05:013
214059975,2cyclictest765-21snmpd20:28:583
214059975,2cyclictest765-21snmpd19:26:033
214059975,2cyclictest765-21snmpd00:33:033
213969977,0cyclictest765-21snmpd23:49:411
213969977,0cyclictest765-21snmpd21:52:161
213969977,0cyclictest765-21snmpd21:31:271
213969977,0cyclictest765-21snmpd21:01:391
213969977,0cyclictest765-21snmpd20:45:111
213969977,0cyclictest763-21lldpd23:55:001
213969977,0cyclictest763-21lldpd22:33:591
213969977,0cyclictest763-21lldpd21:09:131
213969977,0cyclictest506-21avahi-daemon23:01:461
213969977,0cyclictest506-21avahi-daemon22:56:421
213969976,1cyclictest506-21avahi-daemon23:27:031
213969976,1cyclictest506-21avahi-daemon00:10:191
213969976,1cyclictest499-21systemd-resolve22:47:341
213969975,2cyclictest765-21snmpd23:38:401
213969975,2cyclictest765-21snmpd21:36:221
213969975,2cyclictest765-21snmpd00:18:381
213969975,2cyclictest29805-21cat19:35:181
213929977,0cyclictest765-21snmpd21:05:510
213929977,0cyclictest765-21snmpd20:00:520
213929977,0cyclictest765-21snmpd19:12:020
213929977,0cyclictest763-21lldpd22:09:030
213929977,0cyclictest28075-21gltestperf19:30:100
213929976,1cyclictest763-21lldpd22:33:070
213929976,1cyclictest763-21lldpd19:16:580
213929976,1cyclictest739-21ntpd22:18:330
213929976,1cyclictest506-21avahi-daemon23:52:270
213929975,2cyclictest765-21snmpd22:10:140
213929975,2cyclictest765-21snmpd21:46:040
213929975,2cyclictest739-21ntpd23:43:320
213929975,2cyclictest739-21ntpd23:26:410
213929975,2cyclictest739-21ntpd21:54:170
213929975,2cyclictest739-21ntpd21:32:560
213929975,2cyclictest739-21ntpd21:20:520
213929975,2cyclictest0-21swapper/022:47:520
214059966,0cyclictest763-21lldpd23:17:313
214059966,0cyclictest763-21lldpd22:51:263
214059966,0cyclictest763-21lldpd22:38:173
214059965,1cyclictest765-21snmpd20:21:073
214059965,1cyclictest763-21lldpd21:19:493
214059965,1cyclictest499-21systemd-resolve21:42:393
214059965,1cyclictest30657-21cpuspeed_turbos23:10:133
214059964,2cyclictest765-21snmpd23:37:013
214059964,2cyclictest765-21snmpd22:01:373
214059963,2cyclictest7147-21ssh00:00:123
214059963,2cyclictest554-21rs:main3
214059963,2cyclictest554-21rs:main3
214059963,2cyclictest31979-21ssh21:12:323
214059963,2cyclictest254-20systemd-journal23:03:123
214059963,2cyclictest254-20systemd-journal21:53:493
213969966,0cyclictest765-21snmpd20:21:491
213969966,0cyclictest499-21systemd-resolve22:51:581
213969965,1cyclictest765-21snmpd20:37:501
213969965,1cyclictest2588-21gltestperf22:35:181
213969965,1cyclictest24055-21sh00:21:541
213969964,2cyclictest765-21snmpd20:07:191
213969964,2cyclictest765-21snmpd19:21:051
213969964,2cyclictest530-21NetworkManager00:26:471
213969963,2cyclictest26556-21ssh23:44:541
213969963,2cyclictest25860-21ssh22:24:171
213969963,2cyclictest11150-21cron00:05:001
213969962,3cyclictest9258-21ssh23:22:441
213969962,3cyclictest5642-21ssh21:59:381
213969962,3cyclictest1705-21ssh23:14:321
213969960,5cyclictest8622-21ssh21:22:261
213929966,0cyclictest763-21lldpd22:53:390
213929966,0cyclictest763-21lldpd00:21:260
213929965,1cyclictest29556-21fschecks_count19:35:160
213929965,1cyclictest254-20systemd-journal23:30:020
213929965,1cyclictest22784-21needreboot20:50:180
213929964,2cyclictest765-21snmpd23:04:210
213929964,2cyclictest765-21snmpd20:40:200
213929964,2cyclictest765-21snmpd20:16:220
213929964,2cyclictest739-21ntpd21:55:530
213929964,2cyclictest739-21ntpd19:40:010
213929962,3cyclictest11762-21ssh00:05:430
214059955,0cyclictest765-21snmpd21:37:313
214059955,0cyclictest765-21snmpd21:25:093
214059955,0cyclictest28061-21gltestperf19:30:103
214059955,0cyclictest23335-21cpuspeed_turbos19:15:133
214059954,1cyclictest511ktimers/320:00:153
214059953,2cyclictest530-21NetworkManager22:19:413
214059952,2cyclictest9385-21ssh22:43:283
214059951,3cyclictest12764-21ssh00:07:373
213969955,0cyclictest765-21snmpd20:25:581
213969955,0cyclictest506-21avahi-daemon19:53:111
213969955,0cyclictest27439-21munin-plugin-st19:29:591
213969954,1cyclictest506-21avahi-daemon22:06:171
213969954,1cyclictest506-21avahi-daemon19:48:491
213969954,1cyclictest331ktimers/122:10:261
213969953,2cyclictest765-21snmpd20:56:291
213969953,2cyclictest530-21NetworkManager00:33:491
213969952,2cyclictest22392-21ssh21:40:041
213969952,2cyclictest15864-21sh23:31:161
213969951,4cyclictest29771-21uniq21:10:141
213969951,3cyclictest32733-21seq00:35:151
213929955,0cyclictest8013-21tail20:05:200
213929954,1cyclictest8917-21df20:10:160
213929954,1cyclictest7165-21ntpq22:40:200
213929954,1cyclictest12443-21gltestperf20:20:170
213929953,2cyclictest17627-21cat20:35:160
213929952,3cyclictest530-21NetworkManager23:21:300
213929952,3cyclictest530-21NetworkManager19:57:130
213929952,2cyclictest959-21ntpq19:45:170
213929952,2cyclictest554-21rs:main0
213929952,2cyclictest25773-21grep19:20:200
213929952,2cyclictest2364-21grep19:50:170
213929952,2cyclictest10721-21rm21:25:220
214059943,1cyclictest22192-21ntpq19:10:193
214059942,2cyclictest765-21snmpd00:25:103
213969942,2cyclictest765-21snmpd22:15:501
213969942,2cyclictest26791-21kworker/u16:2+nfsiod23:07:111
213969941,2cyclictest3919-21grep19:55:141
213969941,2cyclictest22235-21awk20:50:141
213969940,3cyclictest5898-21grep20:00:181
213969940,3cyclictest23976-21ntp_states19:15:181
213969940,3cyclictest16153-21ntpq20:30:181
213929943,1cyclictest26002-21hddtemp_smartct21:00:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional