You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 04:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Dec 04, 2025 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
862882995349,3cyclictest75-21kswapd019:20:013
862878995048,1cyclictest75-21kswapd019:20:002
862878993734,2cyclictest1075681-21kworker/u8:0+flush-259:023:20:182
862882993429,3cyclictest2698871-21systemd-journal22:30:003
862882993329,3cyclictest894460-21kworker/u8:0+events_unbound19:46:373
862878993327,4cyclictest1023796-21latency_hist21:45:012
862867993226,4cyclictest1201059-21latency_hist00:35:010
862875993127,3cyclictest1096436-21base6422:55:001
862882993027,2cyclictest1075681-21kworker/u8:0+events_unbound23:01:403
862878992925,3cyclictest1131931-21kworker/u8:3+flush-259:023:30:152
862882992825,2cyclictest841228-21kworker/u8:2+events_unbound19:30:513
862882992825,2cyclictest1131931-21kworker/u8:3+events_unbound23:56:563
862878992821,5cyclictest904699-21latency_hist19:50:002
862875992822,4cyclictest2698871-21systemd-journal21:23:391
862867992818,9cyclictest936303-21kworker/u8:1+flush-259:021:10:000
862882992725,2cyclictest909883-21kworker/u8:2+events_unbound20:12:403
862882992724,2cyclictest857659-21kworker/u8:1+events_unbound19:41:133
862882992724,2cyclictest818985-21kworker/u8:0+events_unbound19:11:093
862882992724,2cyclictest1024026-21kworker/u8:4+events_unbound22:31:393
862878992723,3cyclictest936303-21kworker/u8:1+flush-259:020:35:002
862882992623,2cyclictest936304-21kworker/u8:4+events_unbound20:59:393
862882992623,2cyclictest894460-21kworker/u8:0+events_unbound20:45:403
862882992623,2cyclictest1096474-21kworker/u8:2+events_unbound00:20:283
862882992623,2cyclictest1003253-21kworker/u8:0+events_unbound21:31:333
862882992622,3cyclictest1034118-21kworker/u8:1+flush-259:022:35:173
862878992623,2cyclictest1131931-21kworker/u8:3+flush-259:023:50:012
862875992621,4cyclictest899825-21/usr/sbin/munin19:45:131
862882992523,2cyclictest936304-21kworker/u8:4+events_unbound21:00:393
862882992522,2cyclictest997824-21kworker/u8:1+events_unbound21:26:333
862882992522,2cyclictest1018696-21kworker/u8:3+events_unbound21:43:093
862878992520,3cyclictest904577-21kworker/u8:3+flush-259:019:50:172
862878992518,5cyclictest862812-21latency_hist19:10:002
86287599259,12cyclictest0-21swapper/122:40:111
862882992422,1cyclictest894460-21kworker/u8:0+events_unbound21:15:163
862882992421,2cyclictest1137921-21kworker/u8:1+events_unbound23:38:423
862878992420,3cyclictest841228-21kworker/u8:2+flush-259:019:30:162
862878992420,3cyclictest818985-21kworker/u8:0+flush-259:019:15:012
86287599240,18cyclictest1107417-21diskstats23:05:141
862867992420,2cyclictest904718-21true19:50:010
862882992320,2cyclictest894460-21kworker/u8:0+events_unbound19:56:163
862882992320,2cyclictest1131931-21kworker/u8:3+events_unbound00:37:173
862882992320,2cyclictest1075681-21kworker/u8:0+events_unbound23:24:413
862878992317,4cyclictest1132780-21latency_hist23:30:012
862878992316,5cyclictest1112152-21latency_hist23:10:012
86287599234,9cyclictest868073-21cat19:15:011
862875992318,4cyclictest1153760-21apt-get23:50:001
86287599230,18cyclictest982489-21taskset21:03:591
86287599230,13cyclictest1079983-21ld22:35:221
862867992317,4cyclictest2698871-21systemd-journal19:26:030
862882992219,2cyclictest1096474-21kworker/u8:2+events_unbound23:28:413
862878992218,3cyclictest1070780-21/usr/sbin/munin22:30:232
862878992218,3cyclictest1008583-21kworker/u8:2+flush-259:021:50:182
862878992216,5cyclictest2698871-21systemd-journal23:45:002
86287599220,17cyclictest1169988-21seq00:04:291
86287599220,15cyclictest1175454-21taskset00:10:131
86287599220,15cyclictest1044306-21seq22:04:411
86287599220,11cyclictest977402-21modprobe20:59:391
862867992218,3cyclictest909883-21kworker/u8:2+flush-259:020:05:010
86286799220,20cyclictest873271-21munin-run19:20:000
862882992118,3cyclictest1024026-21kworker/u8:4+events_unbound21:49:403
862882992118,2cyclictest930812-21kworker/u8:3+events_unbound20:53:393
862882992118,2cyclictest894460-21kworker/u8:0+events_unbound20:05:293
862882992118,2cyclictest1170287-21kworker/u8:1+events_unbound00:19:173
862882992118,2cyclictest1131931-21kworker/u8:3+events_unbound00:05:043
862882992118,2cyclictest1024026-21kworker/u8:4+events_unbound22:24:573
862882992118,2cyclictest1008583-21kworker/u8:2+events_unbound22:01:403
862882992114,5cyclictest1200191-21ntpq00:30:223
862878992117,3cyclictest841228-21kworker/u8:2+flush-259:019:35:172
862878992116,4cyclictest909883-21kworker/u8:2+flush-259:020:15:152
862875992117,3cyclictest904577-21kworker/u8:3+flush-259:019:55:011
86287599210,16cyclictest935769-21chrt20:16:271
86287599210,14cyclictest1142781-21seq23:36:321
86287599210,12cyclictest894179-21chrt19:37:001
86287599210,11cyclictest924809-21ntpq20:05:231
86287599210,11cyclictest1091102-21taskset22:49:551
862867992118,1cyclictest0-21swapper/021:50:010
862867992115,4cyclictest1076087-21diskstats22:35:150
862882992015,4cyclictest1153963-21gpgv23:50:013
862882992013,1cyclictest191rcu_preempt21:20:203
862878992017,2cyclictest1075681-21kworker/u8:0+flush-259:023:15:172
862878992016,3cyclictest75-21kswapd019:20:022
862878992015,4cyclictest930812-21kworker/u8:3+flush-259:020:25:202
862875992016,3cyclictest997824-21kworker/u8:1+flush-259:021:35:171
86287599200,9cyclictest888963-21chrt19:31:301
86287599200,15cyclictest1054914-21taskset22:13:111
86287599200,14cyclictest1195599-21runrttasks00:26:251
86287599200,12cyclictest1106687-21chrt23:01:311
86287599200,11cyclictest835-21runrttasks20:10:211
86287599200,11cyclictest1117100-21seq23:12:481
86287599200,10cyclictest946109-21seq20:26:431
86287599200,10cyclictest919995-21taskset20:03:081
86287599200,10cyclictest1122107-21taskset23:15:571
86287599200,10cyclictest1028644-21taskset21:46:041
862867992014,5cyclictest1165154-21munin-node00:00:130
862867992014,4cyclictest1164691-21gpgv00:00:010
862882991915,3cyclictest1143739-21kworker/u8:0+events_unbound00:00:543
862882991915,2cyclictest0-21swapper/321:14:593
862878991915,3cyclictest997824-21kworker/u8:1+flush-259:021:40:012
862878991915,2cyclictest998072-21sh21:20:002
862878991913,4cyclictest1086409-21diskstats22:45:132
86287599191,9cyclictest914877-21chrt19:58:391
86287599190,9cyclictest1185357-21seq00:17:471
86287599190,9cyclictest1065065-21runrttasks22:21:031
86287599190,9cyclictest1039078-21taskset21:58:151
86287599190,9cyclictest1008161-21chrt21:27:471
86287599190,8cyclictest879060-21/usr/sbin/munin19:25:191
86287599190,8cyclictest1059326-21ntpq22:15:241
86287599190,18cyclictest962954-21sed20:45:161
86287599190,18cyclictest878815-21awk19:25:011
86287599190,15cyclictest1049779-21chrt22:08:341
86287599190,14cyclictest961801-21seq20:43:111
86287599190,14cyclictest1101754-21chrt22:58:591
86287599190,13cyclictest987630-21chrt21:08:151
86287599190,11cyclictest997958-21chrt21:18:461
86287599190,11cyclictest1190400-21runrttasks00:20:401
86287599190,11cyclictest1159059-21taskset23:51:451
86287599190,11cyclictest1127438-21taskset23:23:161
86287599190,11cyclictest1034023-21chrt21:54:201
86287599190,11cyclictest1023451-21seq21:41:011
86287599190,10cyclictest992742-21chrt21:12:531
86287599190,10cyclictest894673-21/usr/sbin/munin19:40:121
86287599190,10cyclictest872801-21chrt19:15:401
86287599190,10cyclictest835-21runrttasks22:31:091
86287599190,10cyclictest1148563-21taskset23:44:101
86287599190,10cyclictest1137508-21seq23:30:271
86287599190,10cyclictest1132665-21chrt23:28:441
86287599190,10cyclictest1013459-21taskset21:34:311
862882991816,2cyclictest1075681-21kworker/u8:0+events_unbound22:46:563
862882991815,2cyclictest894460-21kworker/u8:0+events_unbound20:41:393
862882991815,2cyclictest1170287-21kworker/u8:1+events_unbound00:13:563
862882991815,2cyclictest1096474-21kworker/u8:2+events_unbound22:56:403
862882991815,2cyclictest1034118-21kworker/u8:1+events_unbound22:14:563
862882991815,2cyclictest1008583-21kworker/u8:2+events_unbound21:51:383
862882991811,5cyclictest1136979-21ntpq23:30:193
862878991815,2cyclictest1070527-21cron22:30:002
86287599180,9cyclictest971959-21seq20:51:111
86287599180,9cyclictest956434-21chrt20:35:571
86287599180,9cyclictest1200228-21ntpq00:30:231
86287599180,8cyclictest1163567-21ntpq23:55:201
86287599180,10cyclictest951269-21taskset20:30:551
86287599180,10cyclictest941076-21seq20:22:551
86287599180,10cyclictest1070309-21chrt22:27:161
862867991813,3cyclictest1096954-21apt-get22:55:120
862882991715,2cyclictest894460-21kworker/u8:0+events_unbound20:27:383
862882991715,2cyclictest857659-21kworker/u8:1+events_unbound19:38:393
862882991715,2cyclictest1008583-21kworker/u8:2+events_unbound21:39:383
862882991714,2cyclictest936304-21kworker/u8:4+events_unbound20:39:573
862882991714,2cyclictest936304-21kworker/u8:4+events_unbound20:21:403
862882991714,2cyclictest1096474-21kworker/u8:2+events_unbound00:29:563
862882991714,2cyclictest1075681-21kworker/u8:0+events_unbound23:15:023
862882991714,2cyclictest1024026-21kworker/u8:4+events_unbound21:58:583
862878991713,2cyclictest879508-21irqcore19:25:152
862878991712,3cyclictest2698871-21systemd-journal21:25:392
86287599170,9cyclictest1205935-21chrt00:36:321
86287599170,8cyclictest1174858-21taskset00:05:541
862867991714,2cyclictest951613-21cron20:35:000
862867991714,2cyclictest930812-21kworker/u8:3+flush-259:020:25:160
862867991714,2cyclictest0-21swapper/021:15:000
862867991713,3cyclictest914999-21sh20:00:000
862867991710,5cyclictest1137900-21latency_hist23:35:010
86288299164,2cyclictest905978-21kthreadcore19:50:173
862882991614,2cyclictest930812-21kworker/u8:3+events_unbound21:05:393
862882991614,2cyclictest894460-21kworker/u8:0+events_unbound20:01:363
862882991614,2cyclictest1075681-21kworker/u8:0+events_unbound23:10:263
862882991614,2cyclictest1008583-21kworker/u8:2+events_unbound22:05:173
862882991613,2cyclictest1090762-21kworker/u8:1+events_unbound22:53:403
862882991613,2cyclictest1075681-21kworker/u8:0+events_unbound22:40:063
862882991613,2cyclictest1049954-21kworker/u8:0+events_unbound22:18:493
862882991612,3cyclictest936303-21kworker/u8:1+events_unbound20:33:393
862882991611,4cyclictest1154449-21apt-get23:50:123
86287899168,6cyclictest1206247-21latency_hist00:40:002
862878991613,2cyclictest987773-21cron21:09:592
862878991613,2cyclictest1048997-21ntp_kernel_err22:05:242
862878991611,4cyclictest920162-21apt-get20:05:002
862878991611,3cyclictest936304-21kworker/u8:4+flush-259:020:35:182
862878991610,4cyclictest988213-21diskstats21:10:142
862867991614,1cyclictest60650irq/133-enp0s29f1:rx-022:14:330
862867991614,1cyclictest60650irq/133-enp0s29f1:rx-022:05:190
862867991614,1cyclictest60650irq/133-enp0s29f1:rx-000:25:260
862867991613,2cyclictest60650irq/133-enp0s29f1:rx-022:40:190
862867991613,2cyclictest60650irq/133-enp0s29f1:rx-022:18:240
862867991613,2cyclictest60650irq/133-enp0s29f1:rx-019:10:180
862867991611,4cyclictest1106094-21ntpq23:00:220
86288299155,1cyclictest0-21swapper/323:40:253
862882991512,2cyclictest818985-21kworker/u8:0+events_unbound19:21:003
862882991512,2cyclictest1107016-21kworker/u8:3+events_unbound23:08:433
86287899159,4cyclictest1195894-21cron00:30:002
86287899158,5cyclictest1143090-21apt-get23:40:002
86287899157,5cyclictest1101855-21basename23:00:002
86287899151,6cyclictest982596-21munin-run21:05:002
862878991512,2cyclictest0-21swapper/220:25:002
862878991512,1cyclictest0-21swapper/221:45:192
86287599150,12cyclictest863061-21latency_hist19:10:001
862867991514,1cyclictest60650irq/133-enp0s29f1:rx-023:40:310
862867991514,1cyclictest60650irq/133-enp0s29f1:rx-000:19:120
862867991513,2cyclictest60650irq/133-enp0s29f1:rx-021:32:220
862867991513,1cyclictest60650irq/133-enp0s29f1:rx-023:51:040
862867991513,1cyclictest60650irq/133-enp0s29f1:rx-023:35:210
862867991513,1cyclictest60650irq/133-enp0s29f1:rx-023:10:180
862867991513,1cyclictest60650irq/133-enp0s29f1:rx-020:05:150
862867991513,1cyclictest1180425-21turbostat00:15:000
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-021:51:370
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-020:55:140
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-020:41:330
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-020:38:370
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-020:19:170
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-019:43:410
862867991512,2cyclictest60650irq/133-enp0s29f1:rx-000:09:510
862867991512,2cyclictest3002-21cron22:05:010
862867991511,3cyclictest904962-21apt-get19:50:120
86286799151,13cyclictest1-21systemd19:33:360
862867991511,2cyclictest930943-21turbostat20:14:590
862867991510,4cyclictest1034360-21apt-get21:55:100
862882991411,2cyclictest909883-21kworker/u8:2+events_unbound20:17:153
862882991411,2cyclictest841228-21kworker/u8:2+events_unbound19:29:233
86287899148,4cyclictest961056-21ntpq20:40:212
86287899147,5cyclictest1060234-21awk22:20:002
862878991410,2cyclictest0-21swapper/200:00:012
86286799149,4cyclictest936343-21apt-get20:20:140
86286799149,4cyclictest2698871-21systemd-journal00:39:590
862867991413,1cyclictest60650irq/133-enp0s29f1:rx-020:50:000
862867991412,1cyclictest60650irq/133-enp0s29f1:rx-023:45:130
862867991412,1cyclictest60650irq/133-enp0s29f1:rx-022:50:340
862867991412,1cyclictest60650irq/133-enp0s29f1:rx-019:35:240
862867991411,2cyclictest60650irq/133-enp0s29f1:rx-022:25:210
862867991411,2cyclictest60650irq/133-enp0s29f1:rx-021:35:160
86287899139,3cyclictest936304-21kworker/u8:4+flush-259:020:55:002
86287899139,3cyclictest1096474-21kworker/u8:2+events_unbound23:10:182
86287899139,2cyclictest0-21swapper/222:40:002
86287899139,2cyclictest0-21swapper/222:05:002
86287899138,4cyclictest1170287-21kworker/u8:1+flush-259:000:15:282
86287899138,3cyclictest0-21swapper/220:14:592
862878991310,1cyclictest0-21swapper/200:30:252
862867991312,1cyclictest60650irq/133-enp0s29f1:rx-022:23:250
862867991312,1cyclictest60650irq/133-enp0s29f1:rx-021:19:480
862867991311,2cyclictest60650irq/133-enp0s29f1:rx-000:24:560
862867991311,1cyclictest60650irq/133-enp0s29f1:rx-023:26:500
862867991311,1cyclictest60650irq/133-enp0s29f1:rx-020:45:240
862867991310,2cyclictest60650irq/133-enp0s29f1:rx-023:05:120
862867991310,2cyclictest60650irq/133-enp0s29f1:rx-022:48:160
862867991310,2cyclictest60650irq/133-enp0s29f1:rx-019:20:230
862867991310,1cyclictest0-21swapper/022:35:000
86287899129,2cyclictest0-21swapper/221:30:262
86287899129,1cyclictest0-21swapper/222:10:282
86287899128,3cyclictest1102183-21cpuspeed_turbos23:00:142
86287899128,2cyclictest0-21swapper/222:00:012
86287899127,3cyclictest0-21swapper/221:24:592
86287899120,3cyclictest899165-21unixbench_singl19:40:252
86286799129,2cyclictest1122788-21cpuspeed_turbos23:20:130
862867991211,1cyclictest61450irq/134-enp0s29f1:tx-023:15:190
862867991211,1cyclictest60650irq/133-enp0s29f1:rx-021:25:280
862867991210,1cyclictest60650irq/133-enp0s29f1:rx-021:40:200
86287899118,2cyclictest1081176-21cpuspeed_turbos22:40:122
86287899118,2cyclictest0-21swapper/222:50:182
86287899117,2cyclictest1175215-21latency_hist00:10:012
86287899114,4cyclictest1189876-21ntpq00:20:222
86286799119,2cyclictest61450irq/134-enp0s29f1:tx-021:20:180
86287899107,2cyclictest1177807-21tr00:10:202
86287899106,2cyclictest0-21swapper/223:50:182
8628789992,5cyclictest914121-21ntpq19:55:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional