You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 23:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Wed Nov 19, 2025 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22057992523,1cyclictest31393-21cron08:35:001
22066992422,2cyclictest13166-21apt-get10:10:023
22066992421,2cyclictest27649-21cron11:45:013
2206199240,23cyclictest21173-21apt-get10:26:562
2205799240,23cyclictest1657-21basename07:35:011
2205299240,0cyclictest0-21swapper/012:31:550
22066992320,2cyclictest20686-21ls09:20:013
2206699230,22cyclictest677-21apt-get09:46:553
2206699230,1cyclictest313-21systemd-journal10:40:003
2206699230,1cyclictest24228-21apt-get08:21:553
22061992320,2cyclictest22122-21turbostat12:40:002
2206199230,22cyclictest6280-21awk08:50:002
2206199230,22cyclictest25488-21turbostat09:30:002
2206199230,22cyclictest17630-21sh11:24:592
22057992320,2cyclictest20774-21apt10:24:591
22057992320,2cyclictest10751-21sh10:05:011
22052992323,0cyclictest0-21swapper/008:56:550
22052992321,1cyclictest313-21systemd-journal08:45:000
22052992320,2cyclictest24404-21sh07:15:000
22066992220,1cyclictest1437-21ls08:40:003
2206699220,21cyclictest401-21apt-get11:56:563
2206699220,21cyclictest3554-21apt-get09:51:563
2206699220,21cyclictest29240-21apt-get08:30:023
2206699220,21cyclictest22106-21run-parts12:40:003
2206699220,21cyclictest0-21swapper/309:11:573
2206699220,1cyclictest6265-21sleep08:50:003
2206699220,1cyclictest28614-21perf08:30:013
22061992220,1cyclictest16925-21awk12:30:012
2206199220,21cyclictest2509-21perf12:00:002
2206199220,21cyclictest0-21swapper/209:41:552
2206199220,1cyclictest13885-21apt-get09:06:552
22057992221,0cyclictest0-21swapper/110:56:591
22057992220,1cyclictest30269-21cron09:40:001
22057992220,1cyclictest12111-21turbostat12:20:001
22057992220,1cyclictest12111-21turbostat12:20:001
2205799220,2cyclictest27871-21cron09:34:591
2205799220,21cyclictest26797-21sendmail07:20:001
2205799220,21cyclictest21800-21apt08:16:551
2205799220,1cyclictest1879-21apt-get08:41:541
2205799220,1cyclictest16936-21cut12:30:011
2205799220,1cyclictest13166-21apt-get10:10:021
22052992220,2cyclictest20027-21cron11:30:000
22052992220,1cyclictest0-21swapper/007:50:000
2205299220,22cyclictest0-21swapper/012:04:200
2205299220,21cyclictest61550irq/126-eno110:50:010
2205299220,21cyclictest28792-21apt-get10:41:560
2205299220,21cyclictest25564-21apt-get10:35:000
2205299220,21cyclictest23257-21apt-get11:36:590
2205299220,21cyclictest21177-21chrt10:26:560
2205299220,21cyclictest0-21swapper/012:10:000
2205299220,21cyclictest0-21swapper/010:59:590
2205299220,21cyclictest0-21swapper/010:11:560
2205299220,21cyclictest0-21swapper/007:35:000
2205299220,1cyclictest313-21systemd-journal08:30:010
2205299220,1cyclictest26807-21cron07:20:000
2206699210,20cyclictest0-21swapper/311:21:573
2206699210,20cyclictest0-21swapper/310:26:563
2206699210,0cyclictest25908-21apt-get09:31:583
2206699210,0cyclictest0-21swapper/311:01:563
2206699210,0cyclictest0-21swapper/307:51:563
22061992121,0cyclictest0-21swapper/208:31:582
22061992120,1cyclictest7309-21turbostat12:09:592
22061992120,1cyclictest3872-21grep08:45:002
22061992120,1cyclictest28382-21perf10:40:002
22061992120,1cyclictest12539-21apt-get12:21:592
22061992119,1cyclictest5637-21munin-run11:00:002
22061992119,1cyclictest30320-21latency_hist09:39:592
22061992119,1cyclictest1480-21awk08:40:002
2206199210,20cyclictest32035-21apt-get07:31:542
2206199210,20cyclictest0-21swapper/211:36:592
2206199210,20cyclictest0-21swapper/210:46:562
2206199210,20cyclictest0-21swapper/209:11:582
2206199210,20cyclictest0-21swapper/208:26:552
2206199210,1cyclictest23110-21perf09:25:002
2206199210,0cyclictest17013-21apt-get08:06:562
2206199210,0cyclictest0-21swapper/207:26:572
22057992121,0cyclictest0-21swapper/111:35:021
22057992121,0cyclictest0-21swapper/110:41:561
22057992120,1cyclictest27243-21apt-get07:21:571
22057992119,1cyclictest5953-21apt-get09:56:561
2205799210,20cyclictest18054-21apt-get11:26:561
2205799210,20cyclictest0-21swapper/109:21:551
2205799210,20cyclictest0-21swapper/108:21:541
2205799210,0cyclictest23578-21apt-get10:31:561
2205799210,0cyclictest0-21swapper/111:41:561
2205799210,0cyclictest0-21swapper/110:46:561
2205799210,0cyclictest0-21swapper/108:06:571
22052992120,0cyclictest13235-21apt-get11:16:550
22052992120,0cyclictest0-21swapper/010:21:570
22052992119,1cyclictest7963-21apt-get10:00:000
22052992119,1cyclictest14184-21ls08:00:000
2205299210,21cyclictest0-21swapper/012:16:580
2205299210,21cyclictest0-21swapper/012:16:580
2205299210,21cyclictest0-21swapper/010:04:260
2205299210,21cyclictest0-21swapper/007:41:540
2205299210,21cyclictest0-21swapper/007:27:560
2205299210,20cyclictest7717-21apt-get12:11:550
2205299210,1cyclictest0-21swapper/011:58:260
2205299210,1cyclictest0-21swapper/010:20:000
2205299210,0cyclictest0-21swapper/011:21:570
2205299210,0cyclictest0-21swapper/009:11:580
22066992020,0cyclictest22426-21apt-get07:11:563
22066992020,0cyclictest20452-21apt-get11:31:573
22066992020,0cyclictest19404-21apt-get08:11:543
22066992020,0cyclictest14935-21apt-get12:26:573
22066992020,0cyclictest11169-21apt-get10:06:563
22066992020,0cyclictest0-21swapper/311:35:023
22066992020,0cyclictest0-21swapper/311:11:563
22066992020,0cyclictest0-21swapper/309:26:573
22066992020,0cyclictest0-21swapper/309:21:553
22066992020,0cyclictest0-21swapper/309:01:573
22066992020,0cyclictest0-21swapper/308:41:553
22066992020,0cyclictest0-21swapper/307:56:563
22066992020,0cyclictest0-21swapper/307:31:553
22066992018,1cyclictest20055-21awk11:30:003
22066992018,1cyclictest0-21swapper/311:55:003
2206699200,2cyclictest313-21systemd-journal12:14:593
2206699200,20cyclictest0-21swapper/308:16:543
2206699200,0cyclictest0-21swapper/310:01:563
22061992020,0cyclictest0-21swapper/211:01:572
22061992020,0cyclictest0-21swapper/210:41:562
22061992020,0cyclictest0-21swapper/210:21:572
22061992019,1cyclictest8446-21apt-get11:06:592
22061992019,1cyclictest30468-21apt-get11:51:582
22061992019,1cyclictest11492-21apt-get09:01:562
22061992019,1cyclictest0-21swapper/210:31:542
22061992018,1cyclictest8599-21apt-get10:00:102
22061992018,1cyclictest5534-21idleruntime-cro09:55:002
2206199200,20cyclictest0-21swapper/207:21:572
2206199200,0cyclictest0-21swapper/211:26:562
2206199200,0cyclictest0-21swapper/209:46:562
22057992020,0cyclictest0-21swapper/112:06:561
22057992020,0cyclictest0-21swapper/111:16:541
22057992020,0cyclictest0-21swapper/110:26:551
22057992020,0cyclictest0-21swapper/110:06:561
22057992020,0cyclictest0-21swapper/107:11:561
22057992019,1cyclictest29639-21apt-get07:26:571
22057992018,1cyclictest30718-21apt-get09:41:561
22057992018,1cyclictest18334-21sh10:20:001
22057992018,1cyclictest12196-21apt-get07:56:561
22057992018,1cyclictest0-21swapper/108:15:001
22057992017,2cyclictest12816-21latency_hist11:14:591
2205799200,0cyclictest1217-21apt-get10:51:561
2205799200,0cyclictest0-21swapper/111:46:581
22052992020,0cyclictest0-21swapper/009:51:570
22052992020,0cyclictest0-21swapper/009:06:550
22052992020,0cyclictest0-21swapper/008:36:540
22052992019,1cyclictest14616-21apt-get08:01:570
22052992018,1cyclictest22463-21basename11:35:000
2205299200,20cyclictest0-21swapper/008:45:280
2205299200,0cyclictest0-21swapper/008:06:570
22066991919,0cyclictest0-21swapper/310:16:573
22066991919,0cyclictest0-21swapper/309:06:553
22066991919,0cyclictest0-21swapper/308:01:573
22066991919,0cyclictest0-21swapper/307:16:553
2206699190,18cyclictest7955-21perf09:59:593
22061991919,0cyclictest0-21swapper/209:56:562
22061991919,0cyclictest0-21swapper/208:21:552
22061991919,0cyclictest0-21swapper/207:56:552
2206199190,19cyclictest0-21swapper/212:31:562
22057991919,0cyclictest4278-21apt-get08:46:561
22057991919,0cyclictest0-21swapper/112:31:561
22057991919,0cyclictest0-21swapper/111:56:551
22057991919,0cyclictest0-21swapper/111:51:581
22057991919,0cyclictest0-21swapper/111:06:581
22057991919,0cyclictest0-21swapper/109:06:551
22057991917,1cyclictest9717-21perf12:15:001
22057991917,1cyclictest0-21swapper/109:55:001
2205799190,0cyclictest6047-21apt-get11:01:561
22052991919,0cyclictest0-21swapper/011:46:560
22052991919,0cyclictest0-21swapper/009:31:580
22052991919,0cyclictest0-21swapper/007:36:570
22052991918,1cyclictest0-21swapper/010:35:020
22052991917,1cyclictest612-21gdbus07:25:000
2205299190,19cyclictest0-21swapper/011:40:040
2205299190,19cyclictest0-21swapper/009:36:320
2205299190,0cyclictest0-21swapper/008:31:030
22066991818,0cyclictest0-21swapper/310:56:593
22066991818,0cyclictest0-21swapper/309:36:563
22066991818,0cyclictest0-21swapper/308:56:553
22066991816,1cyclictest19322-21apt-get12:35:013
22061991818,0cyclictest10837-21apt-get11:11:572
22061991818,0cyclictest0-21swapper/208:16:552
22061991818,0cyclictest0-21swapper/207:41:552
22061991817,0cyclictest9798-21apt-get07:51:562
22061991817,0cyclictest0-21swapper/212:16:582
22061991817,0cyclictest0-21swapper/212:16:582
2206199180,18cyclictest0-21swapper/210:16:582
22057991818,0cyclictest0-21swapper/110:35:021
22057991817,1cyclictest0-21swapper/112:01:551
22057991816,1cyclictest28632-21munin-run08:30:011
22052991818,0cyclictest0-21swapper/011:11:570
22052991818,0cyclictest0-21swapper/011:01:550
22052991816,1cyclictest2764-21perf09:50:000
22052991816,1cyclictest25503-21munin-run09:30:000
2206699173,0cyclictest0-21swapper/312:22:023
22066991717,0cyclictest28073-21apt-get11:46:583
22066991717,0cyclictest0-21swapper/308:51:553
22066991717,0cyclictest0-21swapper/308:06:573
22066991717,0cyclictest0-21swapper/307:41:543
22066991715,1cyclictest29235-21turbostat07:24:593
2206699170,16cyclictest0-21swapper/311:06:593
22061991717,0cyclictest0-21swapper/209:31:582
22061991717,0cyclictest0-21swapper/208:11:552
22057991717,0cyclictest6671-21apt-get08:51:551
22057991717,0cyclictest0-21swapper/109:16:551
22057991717,0cyclictest0-21swapper/108:56:551
22057991717,0cyclictest0-21swapper/108:01:571
22057991717,0cyclictest0-21swapper/107:41:541
22052991717,0cyclictest0-21swapper/009:21:540
22052991716,0cyclictest0-21swapper/012:26:570
22052991713,2cyclictest0-21swapper/009:18:110
22066991616,0cyclictest0-21swapper/312:16:573
22066991616,0cyclictest0-21swapper/312:16:573
22066991616,0cyclictest0-21swapper/310:21:573
22066991616,0cyclictest0-21swapper/307:26:573
22066991615,1cyclictest0-21swapper/307:36:573
2206699160,1cyclictest0-21swapper/307:45:013
22061991615,1cyclictest271rcuc/208:59:592
22061991615,1cyclictest13780-21apt-get10:10:022
2206199160,15cyclictest13223-21munin-run10:10:012
22057991616,0cyclictest7400-21apt-get07:46:551
22057991616,0cyclictest0-21swapper/111:31:581
22057991616,0cyclictest0-21swapper/108:36:551
22052991616,0cyclictest0-21swapper/010:51:560
22052991614,1cyclictest10416-21sh11:10:000
2205299160,16cyclictest0-21swapper/007:51:040
22066991515,0cyclictest0-21swapper/312:06:563
22066991515,0cyclictest0-21swapper/309:41:563
22066991513,1cyclictest0-21swapper/312:01:573
2206699150,1cyclictest0-21swapper/310:46:583
22061991515,0cyclictest0-21swapper/211:31:582
22061991515,0cyclictest0-21swapper/207:11:552
2206199150,1cyclictest0-21swapper/207:37:182
2206199150,14cyclictest0-21swapper/208:01:562
22052991515,0cyclictest0-21swapper/012:36:550
22052991515,0cyclictest0-21swapper/011:51:580
22052991515,0cyclictest0-21swapper/008:11:550
22052991514,1cyclictest21801-21apt-get08:16:550
22052991512,2cyclictest13155-21cron10:10:010
22066991414,0cyclictest0-21swapper/310:40:153
2206699140,14cyclictest0-21swapper/311:15:023
2206699140,0cyclictest0-21swapper/310:51:563
2206699140,0cyclictest0-21swapper/310:30:273
22061991414,0cyclictest0-21swapper/212:00:502
22061991414,0cyclictest0-21swapper/211:41:562
22061991413,1cyclictest0-21swapper/209:16:582
2206199140,14cyclictest0-21swapper/212:10:242
2206199140,14cyclictest0-21swapper/211:45:282
2206199140,14cyclictest0-21swapper/211:19:482
2206199140,14cyclictest0-21swapper/210:50:212
2206199140,14cyclictest0-21swapper/207:45:292
2206199140,14cyclictest0-21swapper/207:16:322
22057991414,0cyclictest0-21swapper/112:36:561
22057991414,0cyclictest0-21swapper/109:10:471
2205799140,14cyclictest0-21swapper/112:22:001
2205799140,14cyclictest0-21swapper/109:25:141
2205799140,14cyclictest0-21swapper/109:00:011
2205799140,0cyclictest0-21swapper/111:20:061
2205799140,0cyclictest0-21swapper/109:46:201
2205799140,0cyclictest0-21swapper/107:35:381
22052991414,0cyclictest0-21swapper/009:00:300
22052991414,0cyclictest0-21swapper/008:51:560
2205299140,14cyclictest0-21swapper/012:22:550
2205299140,14cyclictest0-21swapper/009:41:330
22061991313,0cyclictest0-21swapper/208:52:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional