You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-04-18 - 16:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #8, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack8slot6s.osadl.org (updated Fri Apr 18, 2025 12:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12622992020,0cyclictest4614-21fwupd11:34:412
12622991919,0cyclictest65850irq/126-lan11:24:152
12622991919,0cyclictest65850irq/126-lan10:47:352
12622991919,0cyclictest65850irq/126-lan10:20:162
12622991918,1cyclictest65850irq/126-lan12:34:482
12622991918,1cyclictest65850irq/126-lan10:15:132
12622991918,1cyclictest65850irq/126-lan09:59:342
12622991818,0cyclictest765-21snmpd07:49:312
12622991818,0cyclictest65850irq/126-lan11:49:262
12622991818,0cyclictest65850irq/126-lan11:40:202
12622991818,0cyclictest65850irq/126-lan11:15:162
12622991818,0cyclictest65850irq/126-lan11:11:432
12622991818,0cyclictest65850irq/126-lan10:40:162
12622991818,0cyclictest65850irq/126-lan10:36:052
12622991818,0cyclictest65850irq/126-lan10:25:392
12622991818,0cyclictest65850irq/126-lan10:01:522
12622991818,0cyclictest65850irq/126-lan09:54:022
12622991818,0cyclictest65850irq/126-lan09:38:352
12622991818,0cyclictest65850irq/126-lan09:15:542
12622991818,0cyclictest65850irq/126-lan08:55:182
12622991817,1cyclictest65850irq/126-lan12:35:172
12622991817,1cyclictest65850irq/126-lan12:26:402
12622991817,1cyclictest65850irq/126-lan12:05:442
12622991817,1cyclictest65850irq/126-lan11:55:162
12622991817,1cyclictest65850irq/126-lan11:50:082
12622991817,1cyclictest65850irq/126-lan10:54:542
12622991817,1cyclictest12135-21kworker/2:2+events11:27:332
12622991717,0cyclictest65850irq/126-lan12:16:472
12622991717,0cyclictest65850irq/126-lan12:10:132
12622991717,0cyclictest65850irq/126-lan11:35:132
12622991717,0cyclictest65850irq/126-lan11:09:272
12622991717,0cyclictest65850irq/126-lan10:32:312
12622991717,0cyclictest65850irq/126-lan09:40:122
12622991717,0cyclictest65850irq/126-lan09:29:542
12622991717,0cyclictest65850irq/126-lan07:35:172
12622991716,1cyclictest65850irq/126-lan12:00:192
12622991716,1cyclictest65850irq/126-lan09:47:382
12622991716,1cyclictest65850irq/126-lan08:33:102
12622991616,0cyclictest65850irq/126-lan12:23:232
12622991616,0cyclictest65850irq/126-lan10:55:502
12622991616,0cyclictest65850irq/126-lan10:10:112
12622991616,0cyclictest65850irq/126-lan10:09:032
12622991616,0cyclictest65850irq/126-lan09:19:592
12622991616,0cyclictest65850irq/126-lan08:35:202
12622991616,0cyclictest65850irq/126-lan08:15:232
12622991616,0cyclictest65850irq/126-lan07:30:212
12622991616,0cyclictest65850irq/126-lan07:25:182
12622991616,0cyclictest65850irq/126-lan07:15:152
12622991615,1cyclictest65850irq/126-lan11:01:002
12622991615,1cyclictest65850irq/126-lan09:30:152
12622991615,1cyclictest65850irq/126-lan09:10:112
12622991515,0cyclictest65850irq/126-lan08:45:172
12622991515,0cyclictest65850irq/126-lan08:20:162
12622991515,0cyclictest65850irq/126-lan08:04:352
12622991515,0cyclictest65850irq/126-lan07:20:182
12622991514,1cyclictest65850irq/126-lan08:10:222
12622991414,0cyclictest65850irq/126-lan08:08:092
12622991414,0cyclictest65850irq/126-lan07:40:192
12622991313,0cyclictest65850irq/126-lan08:43:472
12622991111,0cyclictest65850irq/126-lan09:05:182
12622991111,0cyclictest65850irq/126-lan09:00:152
12622991111,0cyclictest65850irq/126-lan07:56:082
12622991111,0cyclictest65850irq/126-lan07:50:232
12622991110,1cyclictest65850irq/126-lan08:50:152
12622991110,1cyclictest65850irq/126-lan07:10:142
126189998,1cyclictest765-21snmpd10:56:011
126189998,1cyclictest765-21snmpd08:23:291
126189998,1cyclictest765-21snmpd07:40:491
126189998,1cyclictest765-21snmpd07:25:521
126189998,1cyclictest506-21avahi-daemon11:32:361
126189998,1cyclictest506-21avahi-daemon11:18:421
126139998,1cyclictest7132-21fschecks_time10:15:150
126139998,1cyclictest506-21avahi-daemon10:09:010
126139994,5cyclictest530-21NetworkManager12:30:470
126269988,0cyclictest765-21snmpd11:56:063
126269988,0cyclictest765-21snmpd10:41:373
126269988,0cyclictest765-21snmpd10:33:363
126269988,0cyclictest763-21lldpd12:16:123
126269988,0cyclictest763-21lldpd10:46:003
126269988,0cyclictest763-21lldpd09:41:233
126269988,0cyclictest763-21lldpd08:46:043
126269987,1cyclictest765-21snmpd08:30:453
126269987,1cyclictest765-21snmpd07:42:343
126269987,1cyclictest763-21lldpd09:55:393
126269987,1cyclictest506-21avahi-daemon09:27:223
126269986,2cyclictest765-21snmpd11:35:533
126269986,2cyclictest765-21snmpd10:20:213
126269986,2cyclictest765-21snmpd10:13:363
126269986,2cyclictest765-21snmpd09:04:043
126269986,2cyclictest765-21snmpd07:50:403
126189988,0cyclictest765-21snmpd09:53:351
126189988,0cyclictest765-21snmpd08:25:591
126189988,0cyclictest763-21lldpd11:52:411
126189987,1cyclictest765-21snmpd10:16:181
126189987,1cyclictest763-21lldpd11:27:471
126189987,1cyclictest763-21lldpd09:02:371
126189987,1cyclictest763-21lldpd08:10:341
126189987,1cyclictest5895-21cut12:40:001
126189987,1cyclictest506-21avahi-daemon11:13:501
126189987,1cyclictest506-21avahi-daemon10:41:531
126189987,1cyclictest499-21systemd-resolve12:28:011
126189987,1cyclictest499-21systemd-resolve11:49:081
126189987,1cyclictest15515-21sh11:45:001
126189986,2cyclictest765-21snmpd10:13:481
126139988,0cyclictest765-21snmpd10:22:390
126139988,0cyclictest765-21snmpd07:20:540
126139988,0cyclictest763-21lldpd11:41:190
126139988,0cyclictest763-21lldpd10:03:440
126139987,1cyclictest765-21snmpd09:14:490
126139987,1cyclictest506-21avahi-daemon10:11:110
126139987,1cyclictest506-21avahi-daemon09:15:200
126139986,2cyclictest765-21snmpd08:57:420
126139986,2cyclictest765-21snmpd08:52:330
126269977,0cyclictest765-21snmpd07:10:123
126269977,0cyclictest763-21lldpd12:27:063
126269977,0cyclictest763-21lldpd12:07:503
126269977,0cyclictest763-21lldpd11:33:213
126269977,0cyclictest763-21lldpd08:00:493
126269976,1cyclictest765-21snmpd09:34:443
126269976,1cyclictest506-21avahi-daemon11:17:303
126269976,1cyclictest506-21avahi-daemon10:18:073
126269976,1cyclictest506-21avahi-daemon07:24:103
126269976,1cyclictest3606-21cron10:55:003
126269975,2cyclictest765-21snmpd12:37:493
126269975,2cyclictest765-21snmpd11:29:413
126269975,2cyclictest765-21snmpd09:38:383
126269975,2cyclictest765-21snmpd09:18:163
126269975,2cyclictest765-21snmpd08:26:053
126269975,2cyclictest24498-21cat11:20:173
126269975,2cyclictest20538-21cat11:50:103
126269972,5cyclictest530-21NetworkManager11:02:083
126229976,1cyclictest12135-21kworker/2:2+events08:25:172
126189977,0cyclictest765-21snmpd12:22:311
126189977,0cyclictest765-21snmpd10:22:511
126189977,0cyclictest765-21snmpd07:19:381
126189977,0cyclictest763-21lldpd11:21:171
126189977,0cyclictest763-21lldpd10:29:011
126189977,0cyclictest763-21lldpd09:59:281
126189977,0cyclictest763-21lldpd09:28:571
126189976,1cyclictest763-21lldpd08:56:591
126189976,1cyclictest506-21avahi-daemon11:58:581
126189976,1cyclictest506-21avahi-daemon09:38:061
126189976,1cyclictest254-20systemd-journal09:30:441
126189976,1cyclictest12633-21idleruntime-cro08:45:001
126189975,2cyclictest765-21snmpd11:00:551
126189975,2cyclictest765-21snmpd07:56:491
126189975,2cyclictest765-21snmpd07:36:491
126189975,2cyclictest765-21snmpd07:14:271
126189973,3cyclictest7266-21ssh11:36:221
126139977,0cyclictest763-21lldpd11:23:290
126139977,0cyclictest763-21lldpd11:11:170
126139977,0cyclictest763-21lldpd10:27:420
126139977,0cyclictest763-21lldpd09:48:450
126139977,0cyclictest763-21lldpd09:00:250
126139977,0cyclictest763-21lldpd08:10:280
126139977,0cyclictest763-21lldpd08:07:300
126139976,1cyclictest765-21snmpd11:35:050
126139976,1cyclictest765-21snmpd11:02:460
126139976,1cyclictest551-21in:imuxsock10:41:170
126139976,1cyclictest506-21avahi-daemon11:56:240
126139976,1cyclictest506-21avahi-daemon07:45:250
126139975,2cyclictest765-21snmpd12:29:250
126139975,2cyclictest765-21snmpd11:50:360
126139975,2cyclictest765-21snmpd10:59:160
126139973,3cyclictest4929-21ssh12:06:330
126139973,3cyclictest30216-21fw_forwarded_lo09:20:160
126269966,0cyclictest765-21snmpd10:36:363
126269966,0cyclictest765-21snmpd08:13:353
126269966,0cyclictest763-21lldpd07:26:313
126269965,1cyclictest506-21avahi-daemon10:27:573
126269965,1cyclictest1-21systemd10:01:413
126269965,1cyclictest10709-21cpuspeed_turbos11:40:153
126269964,2cyclictest765-21snmpd11:45:353
126269964,2cyclictest765-21snmpd09:50:503
126269964,2cyclictest765-21snmpd08:58:423
126269964,2cyclictest765-21snmpd08:07:503
126269962,3cyclictest560-21ssh09:23:483
126269962,3cyclictest20351-21fw_forwarded_lo09:05:143
126269962,3cyclictest11385-21diskstats08:40:143
126189966,0cyclictest765-21snmpd08:51:201
126189966,0cyclictest765-21snmpd08:09:261
126189966,0cyclictest763-21lldpd09:15:451
126189966,0cyclictest763-21lldpd08:30:381
126189966,0cyclictest506-21avahi-daemon10:05:431
126189966,0cyclictest1-21systemd09:45:101
126189965,1cyclictest765-21snmpd10:02:411
126189965,1cyclictest506-21avahi-daemon10:52:391
126189965,1cyclictest499-21systemd-resolve12:01:371
126189965,1cyclictest30483-21cut09:20:181
126189964,2cyclictest765-21snmpd10:35:271
126189964,2cyclictest765-21snmpd09:09:341
126189964,2cyclictest765-21snmpd08:16:111
126189964,2cyclictest765-21snmpd07:54:341
126189963,2cyclictest254-20systemd-journal12:31:551
126189963,2cyclictest15800-21ssh12:16:401
126189963,2cyclictest15222-21ssh09:42:221
126139965,1cyclictest765-21snmpd08:02:170
126139965,1cyclictest765-21snmpd07:33:160
126139965,1cyclictest506-21avahi-daemon12:01:480
126139965,1cyclictest506-21avahi-daemon07:50:190
126139965,1cyclictest24660-21fschecks_count09:55:160
126139965,1cyclictest20076-21needreboot11:15:150
126139964,2cyclictest530-21NetworkManager09:31:460
126139963,3cyclictest530-21NetworkManager11:09:420
126139963,3cyclictest530-21NetworkManager10:36:530
126139963,2cyclictest11227-21ssh12:12:270
126139962,3cyclictest17648-21ssh12:19:000
126269955,0cyclictest763-21lldpd08:22:063
126269955,0cyclictest763-21lldpd07:46:153
126269953,2cyclictest765-21snmpd08:38:573
126269953,2cyclictest765-21snmpd07:57:103
126269953,2cyclictest530-21NetworkManager11:10:153
126269953,2cyclictest530-21NetworkManager11:09:083
126269952,2cyclictest23587-21ssh09:11:463
126269952,2cyclictest21052-21ssh12:21:433
126269951,3cyclictest30968-21ssh12:00:073
126269951,3cyclictest254-20systemd-journal10:55:093
126269951,3cyclictest19947-21hddtemp_smartct07:30:173
126269951,3cyclictest1900-21ssh10:08:153
126189955,0cyclictest765-21snmpd11:06:011
126189955,0cyclictest763-21lldpd08:02:421
126189954,1cyclictest763-21lldpd07:20:531
126189953,2cyclictest765-21snmpd08:47:301
126189953,2cyclictest530-21NetworkManager12:05:081
126189952,2cyclictest20781-21kworker/u16:0+nfsiod10:49:431
126189952,2cyclictest1396-21kworker/u16:2+nfsiod12:10:371
126189951,3cyclictest17945-21memory10:30:111
126139955,0cyclictest765-21snmpd10:51:130
126139955,0cyclictest506-21avahi-daemon09:09:500
126139954,1cyclictest765-21snmpd11:29:110
126139954,1cyclictest765-21snmpd07:27:010
126139954,1cyclictest554-21rs:main0
126139953,2cyclictest765-21snmpd10:48:160
126139953,2cyclictest20498-21cat09:50:110
126139953,2cyclictest1777-21ssh09:25:380
126139953,2cyclictest0-21swapper/012:35:270
126139952,3cyclictest530-21NetworkManager08:29:250
126139952,2cyclictest554-21rs:main0
126139952,2cyclictest554-21rs:main0
126139952,2cyclictest530-21NetworkManager12:20:330
126139952,2cyclictest254-20systemd-journal10:30:510
126139952,2cyclictest254-20systemd-journal08:46:190
126139952,2cyclictest254-20systemd-journal08:35:000
126139952,2cyclictest13957-21ld09:40:170
126139952,2cyclictest10544-21ssh09:36:080
126139950,5cyclictest11878-21ntpq08:40:180
126269943,1cyclictest765-21snmpd08:15:113
126269942,2cyclictest21664-21cat07:35:173
126269941,2cyclictest17097-21rm09:45:073
126269940,4cyclictest8565-21df_inode12:10:083
126269940,4cyclictest15749-21ntpq08:50:213
126269940,3cyclictest28154-21ssh12:30:093
126269940,3cyclictest14661-21diskstats07:15:153
126189941,2cyclictest25175-21sed07:45:151
126189940,4cyclictest282-21systemd-udevd09:10:011
126189940,3cyclictest9259-21diskstats08:35:131
126189940,3cyclictest20274-21ntp_states07:30:211
126139942,2cyclictest739-21ntpd07:15:020
126139942,2cyclictest2925-21netstat08:15:170
126139942,2cyclictest28444-21cat07:55:150
126139941,3cyclictest739-21ntpd08:20:000
126139941,3cyclictest13528-21ntp_states07:10:180
126139941,2cyclictest7881-21uname08:30:190
126139940,3cyclictest21906-21ntpq07:35:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional