You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-21 - 09:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot6s.osadl.org (updated Mon Apr 21, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8351991919,0cyclictest65850irq/126-lan00:37:302
8351991918,1cyclictest65850irq/126-lan23:49:542
8351991918,1cyclictest65850irq/126-lan22:41:032
8351991918,1cyclictest65850irq/126-lan22:39:592
8351991918,1cyclictest65850irq/126-lan21:52:292
8351991918,1cyclictest65850irq/126-lan21:40:172
8351991918,1cyclictest65850irq/126-lan00:29:482
8351991818,0cyclictest65850irq/126-lan23:20:172
8351991818,0cyclictest65850irq/126-lan23:00:552
8351991818,0cyclictest65850irq/126-lan22:27:362
8351991818,0cyclictest65850irq/126-lan22:08:042
8351991818,0cyclictest65850irq/126-lan21:31:362
8351991817,1cyclictest65850irq/126-lan23:35:002
8351991817,1cyclictest65850irq/126-lan23:26:042
8351991817,1cyclictest65850irq/126-lan23:19:162
8351991817,1cyclictest65850irq/126-lan23:10:152
8351991817,1cyclictest65850irq/126-lan22:31:582
8351991817,1cyclictest65850irq/126-lan22:22:472
8351991817,1cyclictest65850irq/126-lan22:10:352
8351991817,1cyclictest65850irq/126-lan21:35:162
8351991817,1cyclictest65850irq/126-lan21:20:382
8351991817,1cyclictest65850irq/126-lan00:21:392
8351991817,1cyclictest65850irq/126-lan00:02:502
8351991717,0cyclictest65850irq/126-lan23:58:432
8351991717,0cyclictest65850irq/126-lan22:56:072
8351991717,0cyclictest65850irq/126-lan22:51:452
8351991717,0cyclictest65850irq/126-lan22:15:192
8351991717,0cyclictest65850irq/126-lan21:56:392
8351991717,0cyclictest65850irq/126-lan21:10:562
8351991717,0cyclictest65850irq/126-lan19:30:282
8351991716,1cyclictest65850irq/126-lan23:50:192
8351991716,1cyclictest65850irq/126-lan23:40:192
8351991716,1cyclictest65850irq/126-lan23:08:552
8351991716,1cyclictest65850irq/126-lan22:45:342
8351991716,1cyclictest65850irq/126-lan21:18:182
8351991716,1cyclictest65850irq/126-lan19:55:182
8351991716,1cyclictest65850irq/126-lan00:19:402
8351991716,1cyclictest65850irq/126-lan00:10:072
8351991616,0cyclictest65850irq/126-lan22:02:532
8351991616,0cyclictest65850irq/126-lan21:45:192
8351991616,0cyclictest65850irq/126-lan20:45:152
8351991616,0cyclictest65850irq/126-lan19:20:152
8351991615,1cyclictest65850irq/126-lan23:35:182
8351991615,1cyclictest65850irq/126-lan21:26:302
8351991615,1cyclictest65850irq/126-lan20:15:192
8351991615,1cyclictest65850irq/126-lan19:25:182
8351991615,1cyclictest65850irq/126-lan00:31:292
8351991515,0cyclictest65850irq/126-lan20:50:172
8351991515,0cyclictest65850irq/126-lan20:25:162
8351991515,0cyclictest65850irq/126-lan19:45:182
8351991515,0cyclictest65850irq/126-lan19:15:152
8351991514,1cyclictest65850irq/126-lan19:50:172
8351991514,1cyclictest65850irq/126-lan00:06:352
8351991414,0cyclictest65850irq/126-lan21:06:022
8351991414,0cyclictest65850irq/126-lan20:05:132
8351991413,1cyclictest65850irq/126-lan21:00:122
8351991413,1cyclictest65850irq/126-lan19:37:452
8351991313,0cyclictest65850irq/126-lan20:20:012
8351991312,1cyclictest65850irq/126-lan20:00:142
8351991212,0cyclictest65850irq/126-lan19:40:222
8351991212,0cyclictest65850irq/126-lan19:13:282
8351991211,1cyclictest65850irq/126-lan20:40:202
8351991211,1cyclictest65850irq/126-lan20:35:212
833999126,4cyclictest506-21avahi-daemon00:10:020
8351991010,0cyclictest65850irq/126-lan20:33:272
833999107,3cyclictest765-21snmpd20:17:510
833999105,5cyclictest765-21snmpd19:20:370
833999104,6cyclictest27674-21cat21:20:150
833999103,3cyclictest12200-21dpkg20:55:000
83549998,1cyclictest765-21snmpd23:17:073
83549998,1cyclictest506-21avahi-daemon00:08:103
83519999,0cyclictest65850irq/126-lan20:55:152
83469998,1cyclictest765-21snmpd23:22:131
83469998,1cyclictest765-21snmpd20:18:151
83469998,1cyclictest765-21snmpd19:13:581
83399997,2cyclictest763-21lldpd23:12:020
83399997,2cyclictest506-21avahi-daemon00:26:100
83399996,3cyclictest765-21snmpd23:17:110
83399996,3cyclictest765-21snmpd21:05:440
83399996,3cyclictest765-21snmpd19:33:400
83399996,3cyclictest499-21systemd-resolve20:35:000
83399994,5cyclictest765-21snmpd00:31:070
83549988,0cyclictest765-21snmpd22:13:433
83549988,0cyclictest765-21snmpd19:55:233
83549988,0cyclictest765-21snmpd00:29:423
83549988,0cyclictest763-21lldpd23:57:053
83549988,0cyclictest763-21lldpd23:20:243
83549988,0cyclictest763-21lldpd21:55:223
83549988,0cyclictest763-21lldpd21:38:513
83549987,1cyclictest765-21snmpd19:13:523
83549987,1cyclictest763-21lldpd22:29:323
83549987,1cyclictest506-21avahi-daemon23:29:323
83549986,2cyclictest765-21snmpd23:33:433
83549986,2cyclictest765-21snmpd20:40:493
83519987,1cyclictest65850irq/126-lan20:10:212
83519987,1cyclictest65850irq/126-lan20:10:202
83469988,0cyclictest765-21snmpd20:43:251
83469988,0cyclictest765-21snmpd20:01:411
83469988,0cyclictest765-21snmpd00:14:091
83469988,0cyclictest765-21snmpd00:03:511
83469988,0cyclictest763-21lldpd23:53:481
83469988,0cyclictest763-21lldpd23:40:361
83469988,0cyclictest763-21lldpd20:23:441
83469988,0cyclictest763-21lldpd19:27:161
83469988,0cyclictest763-21lldpd19:24:341
83469987,1cyclictest763-21lldpd22:04:111
83469987,1cyclictest506-21avahi-daemon23:38:231
83469986,2cyclictest765-21snmpd23:18:411
83469986,2cyclictest765-21snmpd22:41:551
83399988,0cyclictest765-21snmpd22:09:520
83399988,0cyclictest765-21snmpd21:29:550
83399987,1cyclictest765-21snmpd23:23:400
83399987,1cyclictest30407-21ntp_offset22:00:150
83399986,2cyclictest765-21snmpd22:49:280
83399986,2cyclictest765-21snmpd22:18:590
83399985,3cyclictest765-21snmpd23:47:000
83399985,3cyclictest26784-21sh21:56:140
83399984,4cyclictest0-21swapper/021:35:150
83399984,3cyclictest18096-21ntp_states19:35:220
83399983,2cyclictest739-21ntpd23:35:340
83399983,2cyclictest739-21ntpd20:20:210
83399983,2cyclictest554-21rs:main0
83399983,0cyclictest0-21swapper/023:40:150
83399982,3cyclictest739-21ntpd19:40:260
83399982,2cyclictest765-21snmpd23:31:080
83399982,2cyclictest739-21ntpd23:55:120
83399982,2cyclictest554-21rs:main0
83399982,2cyclictest27983-21ls20:05:120
83399982,2cyclictest26084-21ssh21:18:540
83399982,2cyclictest21533-21ssh23:04:470
83399982,2cyclictest21039-21ssh21:13:130
83399982,2cyclictest19591-21ssh00:15:590
83399982,2cyclictest16270-21ssh22:58:460
83399982,2cyclictest1385-21uptime00:35:190
83399982,1cyclictest0-21swapper/021:50:080
83399981,2cyclictest18112-21ssh22:22:560
83399980,3cyclictest3184-21/usr/sbin/munin20:25:180
83399980,3cyclictest22528-21/usr/sbin/munin00:20:160
83399980,2cyclictest530-21NetworkManager20:13:290
83399980,2cyclictest530-21NetworkManager20:13:280
83399980,0cyclictest0-21swapper/019:58:270
83549977,0cyclictest765-21snmpd23:02:183
83549977,0cyclictest765-21snmpd21:18:323
83549977,0cyclictest765-21snmpd20:22:573
83549977,0cyclictest763-21lldpd23:13:133
83549977,0cyclictest763-21lldpd22:59:283
83549977,0cyclictest763-21lldpd21:09:073
83549977,0cyclictest763-21lldpd20:26:413
83549976,1cyclictest765-21snmpd19:26:133
83549976,1cyclictest551-21in:imuxsock22:54:083
83549976,1cyclictest506-21avahi-daemon23:44:123
83549975,2cyclictest765-21snmpd00:34:163
83549974,2cyclictest530-21NetworkManager22:40:013
83469977,0cyclictest765-21snmpd21:50:371
83469977,0cyclictest765-21snmpd21:47:421
83469977,0cyclictest765-21snmpd19:37:021
83469977,0cyclictest765-21snmpd00:15:161
83469977,0cyclictest763-21lldpd22:53:031
83469977,0cyclictest763-21lldpd22:11:321
83469977,0cyclictest763-21lldpd21:11:041
83469977,0cyclictest763-21lldpd20:08:231
83469976,1cyclictest765-21snmpd19:53:201
83469976,1cyclictest506-21avahi-daemon23:07:391
83469976,1cyclictest499-21systemd-resolve00:30:371
83469975,2cyclictest765-21snmpd20:10:401
83469975,2cyclictest765-21snmpd20:10:391
83469973,3cyclictest18358-21ssh22:23:171
83399977,0cyclictest763-21lldpd22:40:240
83399975,2cyclictest5908-21cat22:10:110
83399972,3cyclictest14021-21apt-get19:25:140
83399972,2cyclictest739-21ntpd23:05:100
83399972,2cyclictest739-21ntpd20:55:210
83399971,2cyclictest739-21ntpd00:02:240
83399971,2cyclictest530-21NetworkManager21:03:110
83399971,2cyclictest28006-21ssh22:35:090
83399971,2cyclictest20222-21ssh22:25:460
83399970,3cyclictest9964-21ssh00:05:240
83399970,3cyclictest9573-21/usr/sbin/munin20:45:160
83399970,3cyclictest9031-21/usr/sbin/munin19:10:160
83399970,3cyclictest8994-21ntpq22:50:110
83399970,3cyclictest7781-21/usr/sbin/munin20:40:160
83399970,3cyclictest7114-21ssh23:25:220
83399970,3cyclictest6518-21sed20:35:220
83399970,3cyclictest28802-21/usr/sbin/munin23:50:170
83399970,3cyclictest23309-21ssh22:30:020
83399970,3cyclictest22724-21fw_conntrack19:50:160
83399970,3cyclictest21422-21ntpq19:45:160
83399970,3cyclictest17057-21ssh21:45:230
83549966,0cyclictest765-21snmpd20:09:573
83549966,0cyclictest765-21snmpd00:37:523
83549966,0cyclictest763-21lldpd22:47:213
83549966,0cyclictest763-21lldpd22:08:423
83549966,0cyclictest763-21lldpd21:31:083
83549966,0cyclictest763-21lldpd19:51:423
83549966,0cyclictest763-21lldpd00:13:133
83549966,0cyclictest554-21rs:main3
83549966,0cyclictest25845-21ssh00:24:403
83549965,1cyclictest765-21snmpd19:24:233
83549965,1cyclictest499-21systemd-resolve23:45:513
83549965,1cyclictest499-21systemd-resolve21:46:333
83549964,2cyclictest765-21snmpd22:00:433
83549964,2cyclictest765-21snmpd21:44:033
83549963,3cyclictest530-21NetworkManager22:36:283
83549963,3cyclictest20759-21fw_forwarded_lo21:50:103
83549963,3cyclictest14768-21fw_forwarded_lo21:00:123
83549962,3cyclictest19044-21ssh00:15:213
83469966,0cyclictest765-21snmpd22:28:361
83469966,0cyclictest763-21lldpd21:07:211
83469966,0cyclictest763-21lldpd00:38:081
83469966,0cyclictest22170-21cron19:50:001
83469965,1cyclictest765-21snmpd23:30:081
83469965,1cyclictest765-21snmpd23:03:071
83469965,1cyclictest765-21snmpd19:58:301
83469965,1cyclictest254-20systemd-journal00:20:081
83469964,2cyclictest765-21snmpd22:59:011
83469964,2cyclictest765-21snmpd22:48:331
83469964,2cyclictest765-21snmpd19:18:191
83469963,3cyclictest530-21NetworkManager22:33:341
83469963,3cyclictest31774-21ssh21:25:081
83469963,3cyclictest282-21systemd-udevd21:42:011
83469963,2cyclictest554-21rs:main1
83469963,2cyclictest27724-21ssh23:11:571
83469962,4cyclictest530-21NetworkManager21:00:031
83399960,3cyclictest26478-21ntpq20:00:170
83399960,3cyclictest10885-21ntpq19:15:180
83549955,0cyclictest763-21lldpd23:08:243
83549955,0cyclictest6000-21gltestperf20:35:163
83549954,1cyclictest29667-21cpuspeed_turbos20:10:203
83549954,1cyclictest29667-21cpuspeed_turbos20:10:203
83549953,2cyclictest739-21ntpd23:37:403
83549953,2cyclictest739-21ntpd22:24:223
83549953,2cyclictest739-21ntpd22:15:273
83549953,2cyclictest739-21ntpd19:37:523
83549953,2cyclictest739-21ntpd00:03:083
83549952,3cyclictest23668-21perl22:30:093
83549952,2cyclictest554-21rs:main3
83549952,2cyclictest32417-21ssh21:26:073
83549952,2cyclictest19209-21ssh21:10:333
83549952,2cyclictest13512-21ntpq20:55:183
83549950,4cyclictest31885-21ntpq20:15:153
83549950,4cyclictest16325-21ntpq19:30:213
83469955,0cyclictest765-21snmpd21:30:301
83469955,0cyclictest765-21snmpd20:31:521
83469955,0cyclictest763-21lldpd23:58:201
83469955,0cyclictest506-21avahi-daemon20:58:511
83469955,0cyclictest506-21avahi-daemon00:06:351
83469953,2cyclictest765-21snmpd22:36:111
83469953,2cyclictest22589-21ssh21:15:121
83469953,2cyclictest18342-21kworker/u16:0+nfsiod21:20:511
83469952,3cyclictest530-21NetworkManager22:08:301
83469952,3cyclictest530-21NetworkManager20:50:221
83469952,3cyclictest3395-21seq20:26:411
83469952,2cyclictest7256-21rm23:25:371
83469952,2cyclictest554-21rs:main1
83469952,2cyclictest15923-21grep19:30:181
83469952,2cyclictest11343-21ssh22:15:171
83469950,4cyclictest26031-21ssh23:47:101
83469950,4cyclictest11885-21munin-run21:39:591
83549943,1cyclictest19775-21ldconfig19:40:223
83549942,2cyclictest21801-21seq19:45:303
83549941,3cyclictest10970-21fw_conntrack20:50:193
83549941,2cyclictest10172-21date19:15:003
83549940,4cyclictest9006-21grep20:45:133
83549940,4cyclictest4579-21ntpq20:30:193
83549940,3cyclictest25976-21diskstats20:00:133
83469942,2cyclictest9424-21cat20:45:151
83469941,2cyclictest5942-21expr20:35:171
83469940,3cyclictest19251-21diskstats19:40:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional