You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-13 - 08:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Fri Sep 13, 2024 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
77069994339,3cyclictest289531-21kworker/u8:2+flush-259:023:30:003
77064994239,2cyclictest393239-21kworker/u8:2+flush-259:000:25:232
77059994236,4cyclictest103128-21apt-get19:35:120
77059994135,4cyclictest273584-21apt-get22:20:000
77069994037,2cyclictest134460-21kworker/u8:3+events_unbound20:35:133
77069994034,4cyclictest383004-21latency_hist00:05:013
77064994037,2cyclictest330653-21kworker/u8:1+flush-259:023:50:162
77064994037,2cyclictest217267-21kworker/u8:0+events_unbound22:15:202
77069993835,2cyclictest160696-21kworker/u8:1+events_unbound21:25:213
77064993835,2cyclictest274239-21kworker/u8:4+events_unbound22:20:152
77069993734,3cyclictest82230-21kworker/u8:0+events_unbound19:55:143
77064993633,2cyclictest341390-21kworker/u8:0+flush-259:023:25:202
77069993532,3cyclictest274239-21kworker/u8:4+events_unbound22:42:303
77069993532,2cyclictest408820-21kworker/u8:0+events_unbound00:28:423
77069993531,3cyclictest325753-21apt-get23:10:113
77069993431,2cyclictest160696-21kworker/u8:1+events_unbound21:21:323
77069993431,2cyclictest134460-21kworker/u8:3+events_unbound20:47:313
77069993431,2cyclictest107790-21kworker/u8:2+events_unbound19:45:293
77069993430,3cyclictest238220-21kworker/u8:2+events_unbound21:53:173
77064993431,2cyclictest160696-21kworker/u8:1+flush-259:021:30:202
77064993430,3cyclictest268488-21kworker/u8:1+flush-259:022:30:202
77064993427,2cyclictest134460-21kworker/u8:3+flush-259:020:35:212
77069993330,2cyclictest367101-21kworker/u8:0+events_unbound00:13:333
77069993330,2cyclictest160696-21kworker/u8:1+events_unbound20:55:283
77064993327,5cyclictest138510-21ntpq20:05:202
77062993328,3cyclictest160696-21kworker/u8:1+events_unbound21:50:191
77059993328,3cyclictest176122-21apt-get20:45:120
77069993230,2cyclictest160696-21kworker/u8:1+events_unbound20:43:323
77069993228,3cyclictest82230-21kworker/u8:0+flush-259:019:35:003
77064993229,2cyclictest217267-21kworker/u8:0+flush-259:022:00:182
77062993228,3cyclictest393239-21kworker/u8:2+flush-259:000:15:341
77059993228,3cyclictest403591-21apt-get00:25:030
77069993129,2cyclictest414537-21kworker/u8:4+events_unbound00:38:343
77069993128,2cyclictest289531-21kworker/u8:2+events_unbound23:17:303
77069993128,2cyclictest289531-21kworker/u8:2+events_unbound22:47:303
77069993128,2cyclictest134460-21kworker/u8:3+events_unbound20:53:103
77069993128,2cyclictest134460-21kworker/u8:3+events_unbound20:09:313
77069993128,2cyclictest129115-21kworker/u8:1+events_unbound20:01:313
77064993128,2cyclictest160696-21kworker/u8:1+flush-259:021:00:152
77062993128,2cyclictest2216201-21kworker/u8:2+flush-259:019:25:241
77062993127,3cyclictest274239-21kworker/u8:4+events_unbound22:40:011
77059993125,4cyclictest242704-21latency_hist21:50:000
77064993027,2cyclictest393239-21kworker/u8:2+flush-259:000:15:202
77064993027,2cyclictest268488-21kworker/u8:1+flush-259:022:45:162
77062993026,3cyclictest186365-21kworker/u8:2+events_unbound21:15:131
77062993025,3cyclictest320628-21apt-get23:05:121
77062993024,5cyclictest404276-21apt-get00:25:131
77062993024,4cyclictest305169-21apt-get22:50:131
77062993024,4cyclictest113451-21apt-get19:45:121
77059993026,3cyclictest279431-21apt-get22:25:120
77069992926,2cyclictest268488-21kworker/u8:1+events_unbound22:58:323
77069992926,2cyclictest232453-21kworker/u8:3+events_unbound21:40:263
77059992923,4cyclictest248292-21diskstats21:55:150
77069992825,2cyclictest217267-21kworker/u8:0+events_unbound22:18:193
77069992825,2cyclictest201804-21kworker/u8:4+events_unbound21:12:313
77069992825,2cyclictest160696-21kworker/u8:1+events_unbound21:37:583
77069992825,2cyclictest160696-21kworker/u8:1+events_unbound21:18:143
77064992825,2cyclictest387916-21kworker/u8:1+events_unbound00:20:162
77064992825,2cyclictest356825-21kworker/u8:3+flush-259:023:45:312
77062992824,3cyclictest160696-21kworker/u8:1+flush-259:021:35:211
77069992724,2cyclictest82230-21kworker/u8:0+events_unbound19:52:333
77069992724,2cyclictest408820-21kworker/u8:0+events_unbound00:33:333
77069992724,2cyclictest367101-21kworker/u8:0+events_unbound23:56:333
77069992724,2cyclictest330653-21kworker/u8:1+events_unbound23:53:303
77069992724,2cyclictest134460-21kworker/u8:3+events_unbound21:09:303
77064992724,2cyclictest289531-21kworker/u8:2+flush-259:023:40:152
77064992724,2cyclictest2216201-21kworker/u8:2+flush-259:019:25:202
77064992724,2cyclictest160696-21kworker/u8:1+events_unbound20:55:172
77064992724,2cyclictest107790-21kworker/u8:2+flush-259:019:40:262
77064992723,3cyclictest232453-21kworker/u8:3+events_unbound21:40:222
77062992723,3cyclictest186365-21kworker/u8:2+events_unbound21:10:141
77062992723,3cyclictest108267-21apt-get19:40:111
77059992723,3cyclictest289531-21kworker/u8:2+flush-259:022:55:210
77069992624,2cyclictest393239-21kworker/u8:2+events_unbound00:17:013
77069992623,2cyclictest289531-21kworker/u8:2+events_unbound23:08:303
77069992623,2cyclictest268488-21kworker/u8:1+events_unbound22:23:043
77069992623,2cyclictest2216201-21kworker/u8:2+events_unbound19:17:283
77069992619,5cyclictest320368-21latency_hist23:05:013
77064992623,2cyclictest48421-21kworker/u8:4+flush-259:019:35:012
77064992623,2cyclictest274239-21kworker/u8:4+events_unbound22:40:132
77064992623,2cyclictest2216201-21kworker/u8:2+events_unbound19:10:152
77064992623,2cyclictest217267-21kworker/u8:0+flush-259:022:05:222
77064992622,2cyclictest341390-21kworker/u8:0+flush-259:023:30:202
77062992623,2cyclictest48421-21kworker/u8:4+flush-259:019:50:151
77062992623,2cyclictest129115-21kworker/u8:1+flush-259:020:00:171
77062992621,4cyclictest124218-21apt-get19:55:131
77062992620,4cyclictest191318-21latency_hist21:00:001
77059992621,4cyclictest48421-21kworker/u8:4+flush-259:020:20:270
77059992621,4cyclictest134460-21kworker/u8:3+flush-259:020:45:000
77059992620,4cyclictest160696-21kworker/u8:1+events_unbound21:15:170
77069992523,2cyclictest330653-21kworker/u8:1+events_unbound23:21:033
77069992522,2cyclictest48421-21kworker/u8:4+events_unbound19:42:303
77064992522,2cyclictest48421-21kworker/u8:4+events_unbound19:20:222
77064992522,2cyclictest48421-21kworker/u8:4+events_unbound19:20:222
77064992522,2cyclictest160696-21kworker/u8:1+flush-259:021:25:162
77064992521,3cyclictest2216201-21kworker/u8:2+flush-259:019:15:182
77062992521,3cyclictest378054-21kworker/u8:4+events_unbound00:00:231
77062992521,3cyclictest289531-21kworker/u8:2+flush-259:023:40:201
77062992519,4cyclictest289487-21latency_hist22:35:001
77059992521,3cyclictest48421-21kworker/u8:4+flush-259:019:50:000
77059992520,4cyclictest403705-21gpgv00:25:010
77059992520,4cyclictest134460-21kworker/u8:3+flush-259:020:37:450
77059992518,5cyclictest356800-21latency_hist23:40:000
77069992422,2cyclictest268488-21kworker/u8:1+events_unbound22:26:563
77069992421,2cyclictest232453-21kworker/u8:3+events_unbound21:48:163
77069992418,4cyclictest144585-21gpgv20:15:003
77064992421,2cyclictest82230-21kworker/u8:0+events_unbound19:35:272
77064992420,2cyclictest274239-21kworker/u8:4+flush-259:022:25:252
77064992419,3cyclictest289531-21kworker/u8:2+flush-259:000:05:272
77062992421,2cyclictest330653-21kworker/u8:1+events_unbound23:20:181
77062992421,2cyclictest222440-21apt-get21:30:121
77062992421,2cyclictest160696-21kworker/u8:1+flush-259:020:45:241
77059992420,3cyclictest107790-21kworker/u8:2+flush-259:020:25:260
77059992417,5cyclictest96752-21ntpq19:25:200
77059992417,5cyclictest268442-21latency_hist22:15:010
77069992320,3cyclictest393239-21kworker/u8:2+events_unbound00:24:573
77069992320,2cyclictest155324-21kworker/u8:0+events_unbound20:29:583
77064992320,2cyclictest48421-21kworker/u8:4+events_unbound20:20:202
77064992320,2cyclictest408820-21kworker/u8:0+flush-259:000:30:162
77062992321,1cyclictest107790-21kworker/u8:2+events_unbound20:40:241
77062992320,2cyclictest232453-21kworker/u8:3+flush-259:021:45:211
77062992320,2cyclictest0-21swapper/121:09:591
77062992319,3cyclictest82230-21kworker/u8:0+events_unbound19:35:231
77062992319,3cyclictest315232-21kworker/u8:3+events_unbound23:00:221
77062992319,3cyclictest310306-21apt-get22:55:111
77069992220,2cyclictest217267-21kworker/u8:0+events_unbound21:58:313
77069992219,2cyclictest48421-21kworker/u8:4+events_unbound19:25:023
77069992219,2cyclictest294639-21idleruntime-cro22:40:003
77064992219,2cyclictest48421-21kworker/u8:4+events_unbound20:11:332
77064992218,3cyclictest330653-21kworker/u8:1+events_unbound23:23:262
77062992219,2cyclictest48421-21kworker/u8:4+flush-259:019:20:241
77062992219,2cyclictest48421-21kworker/u8:4+flush-259:019:20:241
77062992219,2cyclictest330653-21kworker/u8:1+flush-259:023:35:251
77062992219,2cyclictest196444-21munin-run21:04:591
77062992219,2cyclictest107790-21kworker/u8:2+flush-259:020:25:241
77062992218,3cyclictest107790-21kworker/u8:2+events_unbound20:13:051
77059992218,3cyclictest289531-21kworker/u8:2+flush-259:023:45:000
77059992218,3cyclictest268488-21kworker/u8:1+flush-259:022:45:210
77059992218,3cyclictest134492-21apt-get20:05:110
77059992217,3cyclictest419477-21latency_hist00:40:010
77059992215,5cyclictest195592-21ntpq21:00:220
77069992119,2cyclictest217267-21kworker/u8:0+events_unbound22:00:023
77069992118,2cyclictest289531-21kworker/u8:2+events_unbound23:40:423
77069992118,2cyclictest107790-21kworker/u8:2+events_unbound20:19:563
77064992118,2cyclictest289531-21kworker/u8:2+events_unbound23:15:152
77064992118,2cyclictest289531-21kworker/u8:2+events_unbound22:55:272
77064992118,2cyclictest201804-21kworker/u8:4+flush-259:021:10:252
77062992118,2cyclictest367101-21kworker/u8:0+flush-259:000:10:231
77062992118,2cyclictest186365-21kworker/u8:2+events_unbound21:22:321
77062992117,3cyclictest217267-21kworker/u8:0+flush-259:022:00:221
77062992114,5cyclictest86445-21ntpq19:15:201
77059992116,4cyclictest145215-21munin-node20:15:090
77059992114,5cyclictest345688-21ntpq23:25:230
77059992114,5cyclictest2304165-21systemd-journal00:15:000
77069992017,2cyclictest48421-21kworker/u8:4+events_unbound19:24:013
77069992017,2cyclictest48421-21kworker/u8:4+events_unbound19:24:013
77069992017,2cyclictest330653-21kworker/u8:1+events_unbound23:34:003
77064992017,2cyclictest367101-21kworker/u8:0+flush-259:023:55:232
77064992016,3cyclictest289531-21kworker/u8:2+events_unbound23:10:262
77064992015,4cyclictest170959-21/usr/sbin/munin20:40:162
77064992015,3cyclictest2304165-21systemd-journal00:39:542
77062992017,2cyclictest387916-21kworker/u8:1+flush-259:000:20:131
77062992017,2cyclictest341390-21kworker/u8:0+events_unbound23:28:191
77062992017,2cyclictest238220-21kworker/u8:2+flush-259:021:55:311
77062992016,3cyclictest367101-21kworker/u8:0+events_unbound23:57:341
77062992016,3cyclictest274239-21kworker/u8:4+events_unbound22:20:211
77062992016,3cyclictest176088-21kworker/u8:4+flush-259:020:50:251
77059992017,2cyclictest160696-21kworker/u8:1+flush-259:020:55:250
77059992015,4cyclictest134460-21kworker/u8:3+flush-259:020:30:210
77059992014,4cyclictest397578-21ntpq00:15:210
77059992014,4cyclictest350763-21ntpq23:30:230
77059992013,5cyclictest2304165-21systemd-journal21:41:310
77059992013,5cyclictest201579-21latency_hist21:10:000
77069991916,2cyclictest330653-21kworker/u8:1+events_unbound23:35:303
77069991916,2cyclictest274239-21kworker/u8:4+events_unbound22:34:333
77069991916,2cyclictest238220-21kworker/u8:2+events_unbound22:07:053
77069991916,2cyclictest217267-21kworker/u8:0+events_unbound22:14:223
77069991916,2cyclictest217267-21kworker/u8:0+events_unbound21:32:553
77069991916,2cyclictest107790-21kworker/u8:2+events_unbound20:23:483
77069991915,3cyclictest82230-21kworker/u8:0+events_unbound19:35:593
77064991916,2cyclictest378054-21kworker/u8:4+events_unbound00:00:562
77064991916,2cyclictest367101-21kworker/u8:0+events_unbound00:10:222
77064991916,2cyclictest330653-21kworker/u8:1+events_unbound23:38:312
77064991916,1cyclictest0-21swapper/220:30:012
77064991914,3cyclictest2304165-21systemd-journal22:36:302
77062991916,2cyclictest82230-21kworker/u8:0+events_unbound20:06:321
77062991916,2cyclictest274239-21kworker/u8:4+flush-259:022:25:211
77062991916,2cyclictest107790-21kworker/u8:2+events_unbound20:15:221
77062991915,3cyclictest289531-21kworker/u8:2+events_unbound23:15:121
77062991915,3cyclictest274239-21kworker/u8:4+events_unbound22:43:311
77062991915,3cyclictest134460-21kworker/u8:3+events_unbound20:30:211
77059991916,2cyclictest48421-21kworker/u8:4+events_unbound19:30:200
77059991912,5cyclictest310048-21latency_hist22:55:020
7705999190,3cyclictest206733-21sh21:14:590
77069991815,2cyclictest330653-21kworker/u8:1+events_unbound23:48:533
77069991815,2cyclictest134460-21kworker/u8:3+events_unbound21:03:473
77064991815,2cyclictest289531-21kworker/u8:2+flush-259:022:50:172
77064991812,4cyclictest117474-21ntpq19:45:222
77062991815,2cyclictest2216201-21kworker/u8:2+events_unbound19:10:251
77062991812,4cyclictest371356-21ntpq23:50:221
77059991815,2cyclictest398473-21kworker/u8:3+flush-259:000:35:000
77059991814,3cyclictest289531-21kworker/u8:2+flush-259:000:00:170
77069991714,2cyclictest367101-21kworker/u8:0+events_unbound00:08:473
77069991714,2cyclictest2216201-21kworker/u8:2+events_unbound19:14:003
77069991714,2cyclictest107790-21kworker/u8:2+events_unbound20:34:553
77064991713,3cyclictest176088-21kworker/u8:4+flush-259:020:50:292
77064991713,3cyclictest160696-21kworker/u8:1+events_unbound20:45:222
77062991714,2cyclictest326774-21kthreadcore23:10:161
77062991714,2cyclictest217267-21kworker/u8:0+events_unbound22:15:251
77062991713,3cyclictest217267-21kworker/u8:0+events_unbound22:10:231
77062991712,4cyclictest414572-21apt-get00:35:131
77062991712,4cyclictest300025-21apt-get22:45:091
77062991712,4cyclictest258381-21/usr/sbin/munin22:05:131
77059991714,2cyclictest335832-21gpgv23:20:000
77059991714,2cyclictest134460-21kworker/u8:3+flush-259:020:10:250
77059991710,5cyclictest258150-21latency_hist22:05:000
77064991613,2cyclictest82230-21kworker/u8:0+events_unbound19:55:242
77064991613,2cyclictest315231-21kworker/u8:0+flush-259:023:05:252
77064991613,2cyclictest238220-21kworker/u8:2+events_unbound21:55:232
77064991613,2cyclictest186365-21kworker/u8:2+events_unbound21:20:152
77064991613,2cyclictest107790-21kworker/u8:2+events_unbound20:30:152
77064991612,3cyclictest315232-21kworker/u8:3+flush-259:023:00:302
77064991612,3cyclictest145240-21apt-get20:15:102
77062991614,2cyclictest367101-21kworker/u8:0+events_unbound00:08:341
77062991612,3cyclictest107790-21kworker/u8:2+events_unbound20:38:101
7705999169,5cyclictest329701-21ntpq23:10:190
77059991614,1cyclictest60650irq/133-enp0s29f1:rx-023:48:190
77059991613,2cyclictest330653-21kworker/u8:1+flush-259:023:25:000
77059991613,2cyclictest268488-21kworker/u8:1+events_unbound22:30:240
77059991613,2cyclictest0-21swapper/019:55:000
77059991612,3cyclictest107790-21kworker/u8:2+flush-259:019:40:220
77059991610,5cyclictest2304165-21systemd-journal22:44:290
7706499159,4cyclictest232427-21latency_hist21:40:002
77064991512,2cyclictest232453-21kworker/u8:3+events_unbound21:47:592
77064991512,2cyclictest134460-21kworker/u8:3+events_unbound21:05:222
77062991511,3cyclictest107790-21kworker/u8:2+events_unbound20:23:051
77059991514,1cyclictest60650irq/133-enp0s29f1:rx-022:39:220
77059991512,2cyclictest60650irq/133-enp0s29f1:rx-023:05:240
77059991512,2cyclictest60650irq/133-enp0s29f1:rx-020:54:480
77059991512,2cyclictest367101-21kworker/u8:0+flush-259:023:55:220
77059991511,3cyclictest82230-21kworker/u8:0+flush-259:019:55:250
77059991511,2cyclictest60650irq/133-enp0s29f1:rx-021:25:100
77064991411,2cyclictest160696-21kworker/u8:1+events_unbound21:50:132
77064991411,2cyclictest129115-21kworker/u8:1+events_unbound20:01:592
77062991411,2cyclictest160696-21kworker/u8:1+flush-259:021:25:231
77062991410,3cyclictest217267-21kworker/u8:0+events_unbound21:42:571
7705999148,4cyclictest216128-21ntpq21:20:200
77059991413,1cyclictest60650irq/133-enp0s29f1:rx-021:52:060
77059991412,2cyclictest60650irq/133-enp0s29f1:rx-000:05:250
77059991412,1cyclictest60650irq/133-enp0s29f1:rx-021:32:070
77059991411,2cyclictest92494-21turbostat19:25:000
77059991411,2cyclictest92494-21turbostat19:25:000
77059991411,2cyclictest60650irq/133-enp0s29f1:rx-023:00:130
77064991310,2cyclictest82230-21kworker/u8:0+events_unbound19:53:372
77062991311,1cyclictest398473-21kworker/u8:3+events_unbound00:30:341
7705999139,3cyclictest263301-21basename22:10:000
77059991311,1cyclictest60650irq/133-enp0s29f1:rx-023:51:120
77059991311,1cyclictest60650irq/133-enp0s29f1:rx-020:00:230
77059991310,2cyclictest279188-21systemd-run22:25:010
7706499128,3cyclictest186365-21kworker/u8:2+flush-259:021:15:102
7706299129,2cyclictest356825-21kworker/u8:3+flush-259:023:45:171
7706299128,3cyclictest341390-21kworker/u8:0+events_unbound23:30:151
7705999128,3cyclictest2216201-21kworker/u8:2+flush-259:019:20:000
7706499117,3cyclictest238220-21kworker/u8:2+flush-259:022:10:552
7706299117,3cyclictest48421-21kworker/u8:4+events_unbound19:30:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional