You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 06:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Thu Apr 25, 2024 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1941499280,1cyclictest7462-21munin-run21:40:001
1941499270,0cyclictest0-21swapper/121:49:361
19420992613,0cyclictest0-21swapper/323:22:343
19414992619,3cyclictest0-21swapper/123:12:471
1941499260,1cyclictest2754-21df_inode23:52:351
1942099250,24cyclictest8619-21diskmemload21:52:133
19418992525,0cyclictest0-21swapper/221:19:312
1941899250,24cyclictest8619-21diskmemload23:44:542
1941899250,24cyclictest8619-21diskmemload23:44:542
1941899250,24cyclictest8619-21diskmemload00:00:542
19414992523,2cyclictest0-21swapper/122:20:321
19414992521,3cyclictest0-21swapper/121:13:471
19414992520,4cyclictest9482-21apt-get19:57:341
1941499250,24cyclictest0-21swapper/121:17:451
1941499250,24cyclictest0-21swapper/100:23:471
19409992523,1cyclictest0-21swapper/021:41:420
1940999250,24cyclictest8619-21diskmemload22:44:170
1940999250,24cyclictest14526-21ssh21:15:440
1940999250,1cyclictest8619-21diskmemload21:30:160
19420992423,0cyclictest0-21swapper/322:30:013
19420992422,1cyclictest9816-21perf22:15:003
1942099240,24cyclictest0-21swapper/321:29:103
1942099240,23cyclictest10431-21ssh21:12:273
1942099240,0cyclictest8619-21diskmemload21:22:423
19418992424,0cyclictest0-21swapper/222:23:132
19418992423,1cyclictest8619-21diskmemload00:26:382
19418992422,1cyclictest28-21ksoftirqd/219:50:012
19418992422,0cyclictest0-21swapper/223:25:442
1941899240,23cyclictest8619-21diskmemload21:42:402
1941899240,23cyclictest10454-21sed00:35:002
1941899240,23cyclictest0-21swapper/221:38:302
19414992421,3cyclictest0-21swapper/123:16:411
19414992421,2cyclictest21629-21ssh21:22:391
19414992416,0cyclictest0-21swapper/121:40:021
19414992415,2cyclictest19277-21latency_hist00:10:001
1941499240,24cyclictest0-21swapper/121:56:011
1941499240,21cyclictest8619-21diskmemload22:46:561
1941499240,21cyclictest0-21swapper/121:53:371
1941499240,0cyclictest0-21swapper/122:37:151
1941499240,0cyclictest0-21swapper/122:00:121
19409992423,1cyclictest0-21swapper/021:26:290
1940999241,2cyclictest1403-21dump-pmu-power19:40:000
1940999240,23cyclictest8619-21diskmemload22:10:410
1940999240,23cyclictest337-21ls23:14:590
1940999240,23cyclictest0-21swapper/000:36:590
1940999240,22cyclictest9944-40logrotate00:00:010
1940999240,1cyclictest8619-21diskmemload23:37:400
1940999240,0cyclictest8619-21diskmemload21:13:280
19420992323,0cyclictest0-21swapper/322:47:093
19420992323,0cyclictest0-21swapper/321:56:153
1942099231,21cyclictest603-21in:imuxsock20:17:313
1942099230,2cyclictest6176-21ssh21:38:053
1942099230,23cyclictest0-21swapper/323:37:553
1942099230,22cyclictest8619-21diskmemload22:55:313
1942099230,22cyclictest8619-21diskmemload22:19:293
1942099230,22cyclictest8619-21diskmemload00:12:333
1942099230,22cyclictest15532-21hddtemp_smartct21:47:403
1942099230,22cyclictest12983-21munin-plugin-st21:45:003
1942099230,22cyclictest0-21swapper/323:32:353
1942099230,1cyclictest14549-21ssh21:15:463
1942099230,0cyclictest8619-21diskmemload21:31:203
1942099230,0cyclictest0-21swapper/323:52:243
1942099230,0cyclictest0-21swapper/300:15:123
19418992323,0cyclictest8619-21diskmemload21:10:042
19418992323,0cyclictest0-21swapper/223:48:152
19418992323,0cyclictest0-21swapper/223:09:262
19418992323,0cyclictest0-21swapper/222:02:152
19418992323,0cyclictest0-21swapper/221:47:532
19418992323,0cyclictest0-21swapper/200:11:142
1941899230,23cyclictest0-21swapper/223:22:542
1941899230,23cyclictest0-21swapper/222:31:562
1941899230,22cyclictest8619-21diskmemload23:15:342
1941899230,0cyclictest8619-21diskmemload21:30:282
1941899230,0cyclictest0-21swapper/223:10:072
1941899230,0cyclictest0-21swapper/222:48:162
1941899230,0cyclictest0-21swapper/222:26:152
1941899230,0cyclictest0-21swapper/221:52:212
1941899230,0cyclictest0-21swapper/221:29:212
19414992323,0cyclictest8619-21diskmemload22:50:501
19414992323,0cyclictest8619-21diskmemload00:16:331
19414992323,0cyclictest0-21swapper/122:57:321
19414992321,1cyclictest0-21swapper/100:34:201
19414992320,3cyclictest0-21swapper/122:11:041
19414992320,3cyclictest0-21swapper/121:26:131
19414992320,3cyclictest0-21swapper/100:37:201
1941499230,23cyclictest0-21swapper/122:41:491
1941499230,23cyclictest0-21swapper/122:34:481
1941499230,22cyclictest5687-21perf00:30:001
1941499230,22cyclictest23880-21apt23:40:011
1941499230,22cyclictest0-21swapper/121:30:011
1941499230,0cyclictest0-21swapper/123:20:271
19409992323,0cyclictest0-21swapper/023:51:470
19409992323,0cyclictest0-21swapper/023:05:430
1940999231,1cyclictest25253-21apt-get22:32:370
1940999230,23cyclictest0-21swapper/023:30:400
1940999230,23cyclictest0-21swapper/022:54:400
1940999230,23cyclictest0-21swapper/022:25:430
1940999230,23cyclictest0-21swapper/000:10:510
1940999230,22cyclictest6253-21apt-get19:50:000
1940999230,22cyclictest19872-21sh21:20:570
1940999230,22cyclictest0-21swapper/022:37:340
1940999230,22cyclictest0-21swapper/021:45:430
1940999230,22cyclictest0-21swapper/020:50:020
1940999230,0cyclictest8619-21diskmemload23:44:300
1940999230,0cyclictest8619-21diskmemload23:44:300
1940999230,0cyclictest8619-21diskmemload22:05:250
1940999230,0cyclictest0-21swapper/023:15:470
1940999230,0cyclictest0-21swapper/021:51:240
19420992222,0cyclictest8619-21diskmemload23:06:043
19420992222,0cyclictest0-21swapper/323:58:303
19420992222,0cyclictest0-21swapper/323:40:293
19420992222,0cyclictest0-21swapper/323:40:293
19420992222,0cyclictest0-21swapper/322:44:023
19420992222,0cyclictest0-21swapper/300:35:413
19420992222,0cyclictest0-21swapper/300:22:543
19420992222,0cyclictest0-21swapper/300:07:273
19420992220,1cyclictest6855-21apt-get19:50:023
19420992220,1cyclictest615-21polkitd20:40:003
19420992220,1cyclictest0-21swapper/323:10:313
1942099220,21cyclictest31278-21dpkg20:50:023
1942099220,21cyclictest11455-21sh23:27:153
1942099220,21cyclictest0-21swapper/323:47:013
1942099220,0cyclictest8619-21diskmemload22:00:393
19418992222,0cyclictest8619-21diskmemload00:05:512
19418992222,0cyclictest0-21swapper/223:54:362
19418992222,0cyclictest0-21swapper/223:35:422
19418992222,0cyclictest0-21swapper/222:09:192
19418992221,1cyclictest23681-21sh21:55:162
19418992220,2cyclictest0-21swapper/223:59:162
19418992220,1cyclictest3229-21lldpd22:40:362
19418992220,1cyclictest31425-21cron19:35:002
19418992220,1cyclictest31013-21ntp_states22:37:432
19418992220,1cyclictest0-21swapper/222:16:032
1941899220,22cyclictest0-21swapper/200:23:502
1941899220,21cyclictest8619-21diskmemload22:11:462
1941899220,21cyclictest3843-21wc21:00:002
1941899220,21cyclictest12093-21ssh22:52:402
1941899220,21cyclictest0-21swapper/222:57:512
1941899220,1cyclictest0-21swapper/221:20:412
1941899220,0cyclictest8619-21diskmemload00:35:122
19414992222,0cyclictest0-21swapper/123:59:491
19414992222,0cyclictest0-21swapper/122:15:381
19414992222,0cyclictest0-21swapper/119:32:341
19414992222,0cyclictest0-21swapper/100:11:521
19414992221,1cyclictest4304-21apt-get21:02:361
19414992220,2cyclictest0-21swapper/100:01:261
19414992220,1cyclictest8620-21cron21:10:001
19414992217,0cyclictest0-21swapper/123:40:081
19414992217,0cyclictest0-21swapper/123:40:081
1941499220,22cyclictest0-21swapper/123:47:561
1941499220,21cyclictest21770-21perf19:15:001
1941499220,21cyclictest17948-21perf20:20:001
1941499220,1cyclictest31887-21apt-get19:37:331
1941499220,1cyclictest16259-21apt-get23:32:361
19409992222,0cyclictest0-21swapper/023:46:200
19409992222,0cyclictest0-21swapper/000:17:220
19409992222,0cyclictest0-21swapper/000:08:200
19409992221,1cyclictest0-21swapper/000:04:410
19409992220,1cyclictest0-21swapper/022:00:130
19409992220,1cyclictest0-21swapper/020:55:010
1940999221,21cyclictest62150irq/126-eno123:00:100
1940999220,22cyclictest0-21swapper/021:55:290
1940999220,21cyclictest62150irq/126-eno100:25:010
1940999220,21cyclictest0-21swapper/022:57:350
1940999220,21cyclictest0-21swapper/022:48:110
1940999220,21cyclictest0-21swapper/022:17:230
1940999220,21cyclictest0-21swapper/020:42:350
1940999220,21cyclictest0-21swapper/019:42:350
1940999220,21cyclictest0-21swapper/019:17:360
1940999220,1cyclictest0-21swapper/000:30:580
1940999220,0cyclictest0-21swapper/023:21:350
1940999220,0cyclictest0-21swapper/022:20:340
19420992121,0cyclictest0-21swapper/322:25:323
19420992121,0cyclictest0-21swapper/322:07:253
19420992121,0cyclictest0-21swapper/300:26:033
19420992120,1cyclictest12068-21apt-get20:00:023
19420992118,2cyclictest28496-21sh22:35:023
1942099210,21cyclictest0-21swapper/323:02:063
1942099210,21cyclictest0-21swapper/322:53:463
1942099210,21cyclictest0-21swapper/322:20:133
1942099210,21cyclictest0-21swapper/300:30:513
1942099210,21cyclictest0-21swapper/300:00:223
1942099210,20cyclictest341-21apt-get23:15:013
1942099210,20cyclictest0-21swapper/319:37:343
19418992121,0cyclictest0-21swapper/220:07:312
19418992121,0cyclictest0-21swapper/200:16:482
19418992119,2cyclictest0-21swapper/223:30:142
19418992119,1cyclictest26256-21apt-get20:42:352
1941899210,21cyclictest0-21swapper/223:00:312
1941899210,20cyclictest6698-21apt-get21:07:372
1941899210,0cyclictest0-21swapper/220:00:022
1941899210,0cyclictest0-21swapper/219:22:362
19414992121,0cyclictest0-21swapper/119:47:421
19414992120,0cyclictest1885-21apt-get20:57:361
19414992118,3cyclictest0-21swapper/120:42:351
1941499210,21cyclictest0-21swapper/123:27:401
1941499210,21cyclictest0-21swapper/122:05:421
1941499210,20cyclictest0-21swapper/120:07:311
1941499210,0cyclictest0-21swapper/123:01:331
19409992121,0cyclictest0-21swapper/023:25:130
19409992120,0cyclictest21442-21apt-get20:32:350
19409992120,0cyclictest21442-21apt-get20:32:350
1940999210,0cyclictest2534-21ssh21:35:080
1940999210,0cyclictest0-21swapper/000:20:210
19420992020,0cyclictest0-21swapper/320:42:353
19420992020,0cyclictest0-21swapper/319:57:343
19420992020,0cyclictest0-21swapper/319:42:353
19420992020,0cyclictest0-21swapper/319:12:343
1942099200,0cyclictest12935-21apt-get20:07:313
1942099200,0cyclictest0-21swapper/319:47:423
1941899203,1cyclictest151rcu_preempt21:05:012
19418992020,0cyclictest0-21swapper/219:27:362
19418992018,1cyclictest25773-21sendmail20:40:002
19418992018,1cyclictest1011-21apt20:55:002
19414992020,0cyclictest0-21swapper/120:00:031
19414992020,0cyclictest0-21swapper/119:22:351
19414992020,0cyclictest0-21swapper/119:17:361
1941499200,0cyclictest0-21swapper/120:37:371
1941499200,0cyclictest0-21swapper/119:42:351
19409992020,0cyclictest0-21swapper/021:02:360
1940999200,20cyclictest0-21swapper/019:22:360
19420991917,1cyclictest1648-21apt-get20:55:013
19418991919,0cyclictest0-21swapper/220:32:352
19418991919,0cyclictest0-21swapper/220:32:352
19418991917,1cyclictest0-21swapper/219:50:002
1941899190,18cyclictest0-21swapper/219:57:342
19414991919,0cyclictest31509-21apt-get20:52:371
19414991917,1cyclictest23385-21cron20:35:001
19414991917,1cyclictest23385-21cron20:35:001
19420991818,0cyclictest0-21swapper/320:47:373
19420991818,0cyclictest0-21swapper/320:32:353
19420991818,0cyclictest0-21swapper/320:32:353
19418991816,1cyclictest0-21swapper/219:15:002
19414991818,0cyclictest0-21swapper/119:50:021
19409991818,0cyclictest0-21swapper/020:47:370
19409991818,0cyclictest0-21swapper/020:07:310
19409991818,0cyclictest0-21swapper/019:27:350
19409991815,2cyclictest618-21gdbus20:04:590
19420991715,1cyclictest0-21swapper/321:02:363
19409991717,0cyclictest11448-21sendmail-msp20:00:000
19409991715,1cyclictest0-21swapper/021:07:370
19409991715,1cyclictest0-21swapper/020:37:370
19420991614,1cyclictest31496-21munin-run19:35:003
19418991615,1cyclictest18562-21cron20:25:002
19418991614,1cyclictest0-21swapper/220:47:472
19414991615,0cyclictest19034-21apt-get20:27:321
19409991616,0cyclictest0-21swapper/019:52:360
19409991615,0cyclictest15972-21apt-get20:17:310
19409991614,1cyclictest19822-21apt-get19:12:360
19420991515,0cyclictest24635-21apt-get19:22:353
19420991414,0cyclictest0-21swapper/319:15:373
1942099140,14cyclictest0-21swapper/321:06:023
1942099140,14cyclictest0-21swapper/320:26:253
1942099140,14cyclictest0-21swapper/320:21:063
1942099140,14cyclictest0-21swapper/320:13:233
1942099140,14cyclictest0-21swapper/319:25:083
1941899140,14cyclictest0-21swapper/220:25:322
1941899140,14cyclictest0-21swapper/220:12:232
1941899140,14cyclictest0-21swapper/219:35:442
1941899140,14cyclictest0-21swapper/219:15:212
1941899140,0cyclictest0-21swapper/220:15:512
1941899140,0cyclictest0-21swapper/219:41:002
19414991414,0cyclictest0-21swapper/120:46:181
1941499140,14cyclictest0-21swapper/120:21:291
1941499140,14cyclictest0-21swapper/120:10:021
1941499140,14cyclictest0-21swapper/119:25:181
19409991414,0cyclictest0-21swapper/020:10:100
1940999140,14cyclictest0-21swapper/020:25:060
1940999140,14cyclictest0-21swapper/019:30:080
1940999140,0cyclictest0-21swapper/020:20:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional