You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 07:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Sep 06, 2026 00:45:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
921614998517,32cyclictest1004091-21latency_hist23:55:020
921616998320,20cyclictest984970-21ssh23:13:411
921616998121,19cyclictest959172-21ssh22:17:251
921616998120,23cyclictest979990-21ssh23:02:351
921616998021,23cyclictest1003509-21ssh23:53:591
921616997927,16cyclictest977638-21ssh22:57:271
921616997927,16cyclictest977638-21ssh22:57:271
921616997822,17cyclictest922847-21munin-run19:35:011
921616997822,17cyclictest922847-21munin-run19:35:011
921614997826,5cyclictest16850irq/134-800000000:00:050
921616997622,17cyclictest986732-21ssh23:17:231
921616997622,17cyclictest986732-21ssh23:17:231
921614997626,13cyclictest947766-21ssh21:52:320
921616997427,14cyclictest992032-21sh23:28:521
921616997422,15cyclictest998438-21ssh23:42:531
921616997421,16cyclictest928902-21ssh21:12:011
921616997421,15cyclictest961552-21ssh22:22:251
921616997419,21cyclictest971332-21ssh22:43:501
921616997419,15cyclictest1007096-21ssh00:01:481
921616997418,22cyclictest938578-21ssh21:32:501
92161499742,37cyclictest939428-21ssh21:34:460
921616997321,16cyclictest1014609-21ssh00:18:411
921616997318,20cyclictest1010397-21ssh00:09:231
92161999721,45cyclictest967262-21ssh22:35:022
921616997217,17cyclictest952045-21ssh22:01:481
921616997131,14cyclictest965988-21ssh22:32:081
921616997129,12cyclictest982151-21ssh23:07:151
921616997122,15cyclictest972901-21ssh22:47:061
921616997120,14cyclictest935326-21ssh21:25:361
921616997120,14cyclictest935326-21ssh21:25:351
921616997118,15cyclictest996852-21ssh23:39:351
921616997117,15cyclictest1022568-21ssh00:35:581
921614997127,10cyclictest934724-21ssh21:24:490
921616997019,15cyclictest940492-21ssh21:36:481
921616997019,15cyclictest940492-21ssh21:36:471
921616997018,18cyclictest950398-21ssh21:58:211
921616997018,15cyclictest964732-21ssh22:29:411
921616997017,15cyclictest957166-21ssh22:13:051
921614997033,28cyclictest981652-21kworker/u8:023:20:020
921614997033,28cyclictest981652-21kworker/u8:023:20:020
921616996921,14cyclictest992756-21sshd23:30:071
921616996920,15cyclictest968972-21ssh22:38:501
921616996918,14cyclictest942958-21ssh21:42:091
921616996917,16cyclictest364-21snmpd22:07:011
921616996917,14cyclictest934638-21ssh21:24:281
921616996821,14cyclictest988374-21ssh23:20:351
921616996819,14cyclictest1004835-21ssh23:56:301
921616996818,14cyclictest922341-21munin-run19:25:001
921616996816,16cyclictest947226-21ssh21:51:141
921614996827,24cyclictest1015177-21ssh00:20:020
921616996720,13cyclictest924950-21date20:15:001
921616996717,15cyclictest1011004-21ssh00:10:221
921616996716,16cyclictest1019597-21ssh00:29:511
921616996716,14cyclictest999946-21ssh23:45:541
921616996716,14cyclictest931282-21ssh21:17:101
921614996711,29cyclictest16850irq/134-800000021:50:020
921616996622,15cyclictest945194-21ssh21:46:591
921616996617,14cyclictest927795-21munin-run21:10:001
921616996615,15cyclictest1015977-21ssh00:21:321
921616996614,15cyclictest976273-21ssh22:54:381
921616996421,14cyclictest0-21swapper/100:33:001
92161499641,4cyclictest0-21swapper/023:15:030
921616996320,15cyclictest924745-21cat20:10:021
921616996040,16cyclictest924397-21sshd20:04:051
921614996039,6cyclictest12-21ksoftirqd/023:58:540
921614996038,19cyclictest0-21swapper/019:15:020
921619995849,6cyclictest926758-21kworker/2:300:03:352
921614995828,14cyclictest964313-21ssh22:28:340
921616995610,14cyclictest0-21swapper/120:26:401
921614995628,12cyclictest976973-21ssh22:55:520
921614995628,12cyclictest976973-21ssh22:55:520
921614995628,12cyclictest971516-21ssh22:44:130
921619995545,6cyclictest926758-21kworker/2:323:20:022
921619995545,6cyclictest926758-21kworker/2:323:20:022
921616995539,13cyclictest3467602-21systemd-journal19:19:491
92161499551,17cyclictest951102-21sendmail22:00:000
921619995445,5cyclictest926758-21kworker/2:300:11:122
921619995444,6cyclictest922305-21kworker/2:021:10:022
921616995438,13cyclictest927215-21sshd20:55:051
921614995428,11cyclictest972764-21ssh22:46:350
92161499542,42cyclictest953974-21ssh22:06:040
92161499541,37cyclictest998268-21ssh23:42:310
921619995344,6cyclictest926758-21kworker/2:323:45:012
921619995344,5cyclictest922305-21kworker/2:021:00:022
921619995335,13cyclictest3467602-21systemd-journal23:00:032
921616995313,11cyclictest926505-21systemd-run20:45:011
921614995343,7cyclictest12-21ksoftirqd/022:23:460
921624995236,13cyclictest978739-21systemctl23:00:003
921624995236,13cyclictest978739-21systemctl23:00:003
921619995244,5cyclictest926758-21kworker/2:321:54:142
921619995244,5cyclictest926758-21kworker/2:321:49:292
921619995242,6cyclictest926758-21kworker/2:300:30:012
921614995230,18cyclictest12-21ksoftirqd/022:35:010
921614995227,9cyclictest1011591-21ssh00:11:550
921614995226,12cyclictest16850irq/134-800000023:36:530
921614995224,13cyclictest931562-21ssh21:17:450
92161499522,36cyclictest151rcuc/021:45:020
921614995214,34cyclictest16850irq/134-800000022:20:020
921614995213,34cyclictest16850irq/134-800000021:10:000
921624995131,16cyclictest275-21dbus-daemon22:10:013
921624995113,6cyclictest131rcu_preempt21:52:393
921619995143,5cyclictest934853-21kworker/2:122:14:112
921619995143,5cyclictest926758-21kworker/2:322:58:052
921619995143,5cyclictest926758-21kworker/2:322:58:052
921619995143,5cyclictest926758-21kworker/2:300:20:162
921619995134,13cyclictest1-21systemd23:35:002
921616995137,10cyclictest3467602-21systemd-journal19:54:051
921619995044,3cyclictest926758-21kworker/2:321:16:352
921619995043,4cyclictest926758-21kworker/2:300:31:382
921619995042,5cyclictest926758-21kworker/2:300:38:052
921619995036,8cyclictest1006213-21sendmail00:00:002
92161999502,6cyclictest972683-21ssh22:46:232
921616995036,9cyclictest48-21kcompactd020:48:311
921616995036,9cyclictest48-21kcompactd020:48:301
921616995036,9cyclictest48-21kcompactd019:57:291
921614995043,4cyclictest16950irq/224-800000019:40:000
921614995029,17cyclictest16850irq/134-800000021:30:000
921614995029,17cyclictest16850irq/134-800000021:30:000
92161499502,4cyclictest131rcu_preempt22:15:000
921614995022,13cyclictest1018418-21ssh00:26:540
921624994931,13cyclictest987943-21cat23:20:023
921624994931,13cyclictest987943-21cat23:20:023
921619994941,5cyclictest934853-21kworker/2:122:43:522
921619994941,5cyclictest934853-21kworker/2:122:27:082
921619994941,5cyclictest934853-21kworker/2:122:22:042
921619994941,5cyclictest934853-21kworker/2:121:59:062
921619994941,5cyclictest926758-21kworker/2:321:45:002
921619994940,5cyclictest934853-21kworker/2:122:05:292
921619994940,5cyclictest934853-21kworker/2:122:00:232
921619994940,5cyclictest679246-21kworker/2:219:10:022
921619994913,6cyclictest131rcu_preempt20:16:592
921616994939,6cyclictest925462-21munin-run20:25:011
921616994936,9cyclictest48-21kcompactd021:02:571
921616994936,9cyclictest48-21kcompactd021:02:561
921616994936,10cyclictest923631-21munin-run19:50:011
921616994936,10cyclictest923631-21munin-run19:50:011
921616994936,10cyclictest364-21snmpd19:35:171
921614994926,7cyclictest991493-21ssh23:27:430
921614994923,11cyclictest979900-21ssh23:02:240
921614994918,7cyclictest12-21ksoftirqd/000:08:420
92161499491,32cyclictest969110-21ssh22:39:080
92162499483,3cyclictest150-21pr/ttyS223:38:163
921619994840,5cyclictest926758-21kworker/2:323:53:352
921619994840,5cyclictest926758-21kworker/2:323:07:302
921616994837,8cyclictest48-21kcompactd020:36:181
921616994837,8cyclictest364-21snmpd20:16:381
921616994837,8cyclictest3467602-21systemd-journal19:14:151
921616994835,9cyclictest364-21snmpd19:43:061
921616994830,14cyclictest3467602-21systemd-journal20:33:371
921614994841,4cyclictest16950irq/224-800000023:47:140
921614994837,8cyclictest12-21ksoftirqd/020:15:010
921614994824,20cyclictest940855-21ssh21:37:430
921614994824,20cyclictest940855-21ssh21:37:430
921614994823,10cyclictest3467602-21systemd-journal00:30:530
92161499482,12cyclictest990140-21ls23:25:010
921614994816,28cyclictest12-21ksoftirqd/022:53:450
921624994731,13cyclictest275-21dbus-daemon00:00:033
92162499472,11cyclictest150-21pr/ttyS222:00:533
921619994742,3cyclictest926758-21kworker/2:321:23:132
921619994739,5cyclictest934853-21kworker/2:122:15:582
921619994739,5cyclictest926758-21kworker/2:323:47:252
921619994738,5cyclictest926758-21kworker/2:321:28:242
921619994738,5cyclictest926758-21kworker/2:321:28:242
92161999471,3cyclictest131rcu_preempt21:39:532
92161999471,3cyclictest131rcu_preempt21:39:532
921616994733,10cyclictest364-21snmpd19:27:431
921614994737,6cyclictest12-21ksoftirqd/023:33:080
921614994728,16cyclictest982459-21rm23:07:540
921614994720,23cyclictest1023553-21ssh00:38:150
921624994635,8cyclictest37-21ksoftirqd/322:19:003
921619994638,5cyclictest934853-21kworker/2:122:52:482
921619994638,5cyclictest934853-21kworker/2:122:38:462
921619994638,5cyclictest926758-21kworker/2:300:16:352
921619994638,5cyclictest926758-21kworker/2:300:06:292
921619994638,5cyclictest922305-21kworker/2:019:42:042
921619994637,6cyclictest922305-21kworker/2:020:55:002
921619994637,6cyclictest922305-21kworker/2:019:40:012
921619994631,11cyclictest997111-21sh23:40:012
921614994632,10cyclictest275-21dbus-daemon20:40:010
921614994619,12cyclictest930054-21ssh21:14:520
92162499454,3cyclictest131rcu_preempt20:05:013
92162499453,4cyclictest131rcu_preempt21:30:003
92162499453,4cyclictest131rcu_preempt21:30:003
92162499453,3cyclictest131rcu_preempt23:42:343
921624994530,12cyclictest275-21dbus-daemon00:20:013
921624994529,13cyclictest275-21dbus-daemon21:50:013
921619994538,5cyclictest922305-21kworker/2:021:10:562
921619994537,5cyclictest926758-21kworker/2:323:27:412
921619994537,5cyclictest922305-21kworker/2:020:05:552
921619994537,5cyclictest922305-21kworker/2:019:56:002
92161999453,5cyclictest364-21snmpd21:32:302
921616994532,9cyclictest364-21snmpd20:52:261
921614994528,12cyclictest275-21dbus-daemon20:30:010
921614994519,12cyclictest952400-21ssh22:02:450
92162499445,4cyclictest150-21pr/ttyS223:30:223
92162499444,4cyclictest150-21pr/ttyS220:40:013
92162499444,3cyclictest131rcu_preempt21:16:323
92162499443,3cyclictest150-21pr/ttyS200:10:343
921624994433,6cyclictest309-21in:imuxsock21:15:013
92162499442,8cyclictest965335-21ssh22:30:413
921624994416,7cyclictest131rcu_preempt22:45:023
921619994436,5cyclictest926758-21kworker/2:323:12:052
921619994436,5cyclictest922305-21kworker/2:019:55:002
921619994436,5cyclictest922305-21kworker/2:019:34:032
921619994436,5cyclictest922305-21kworker/2:019:34:022
921619994435,6cyclictest923598-21kworker/2:120:40:022
921616994424,14cyclictest921688-21latency_hist19:10:021
921624994329,10cyclictest3467602-21systemd-journal23:15:013
92162499431,11cyclictest941566-21ssh21:39:223
92162499431,11cyclictest941566-21ssh21:39:223
92161499433,4cyclictest922060-21modprobe19:20:000
921614994318,22cyclictest1017088-21ssh00:24:020
92161499431,10cyclictest925186-21munin-run20:20:010
92162499424,3cyclictest131rcu_preempt23:46:563
921624994228,10cyclictest1-21systemd00:35:013
921624994227,10cyclictest275-21dbus-daemon23:25:003
92162499422,4cyclictest131rcu_preempt22:27:493
921624994218,4cyclictest37-21ksoftirqd/323:09:123
92162499421,10cyclictest922346-21systemd-run19:24:593
921619994234,5cyclictest923598-21kworker/2:120:31:052
921619994233,5cyclictest922305-21kworker/2:021:00:202
921619994233,5cyclictest922305-21kworker/2:021:00:192
921619994230,8cyclictest925695-21sshd20:29:052
921614994226,13cyclictest924110-21sshd19:57:230
921614994217,10cyclictest12-21ksoftirqd/019:30:000
92162499413,3cyclictest131rcu_preempt21:59:043
921624994131,7cyclictest922821-21sshd19:32:203
921624994131,7cyclictest922821-21sshd19:32:193
921624994129,9cyclictest1019695-21systemd-run00:30:013
921624994127,11cyclictest37-21ksoftirqd/323:28:553
92162499412,3cyclictest150-21pr/ttyS221:33:033
921624994116,4cyclictest37-21ksoftirqd/300:00:303
92162499411,3cyclictest131rcu_preempt22:39:293
92161999416,7cyclictest30-21ksoftirqd/219:46:062
92161999416,7cyclictest30-21ksoftirqd/219:46:062
92161999414,3cyclictest131rcu_preempt23:23:372
921619994125,11cyclictest30-21ksoftirqd/219:20:002
92161499417,20cyclictest921606-21cyclictest20:10:010
921614994123,13cyclictest275-21dbus-daemon21:00:010
921614994115,22cyclictest12-21ksoftirqd/021:05:020
921614994115,22cyclictest12-21ksoftirqd/021:05:020
921614994114,18cyclictest0-21swapper/019:10:010
92162499406,3cyclictest150-21pr/ttyS223:01:163
92162499406,3cyclictest131rcu_preempt20:17:053
92162499404,3cyclictest131rcu_preempt20:55:003
92162499403,3cyclictest131rcu_preempt23:51:103
921624994029,9cyclictest37-21ksoftirqd/320:32:313
921624994028,9cyclictest37-21ksoftirqd/322:47:153
921624994026,11cyclictest923890-21systemd-run19:55:003
92162499402,4cyclictest131rcu_preempt19:45:003
92162499402,3cyclictest131rcu_preempt21:05:023
92162499402,3cyclictest131rcu_preempt21:05:023
921624994022,13cyclictest275-21dbus-daemon19:50:013
921624994022,13cyclictest275-21dbus-daemon19:50:003
92161999404,4cyclictest131rcu_preempt20:05:002
92161999402,4cyclictest131rcu_preempt19:15:002
921614994026,10cyclictest275-21dbus-daemon19:35:000
921614994026,10cyclictest275-21dbus-daemon19:35:000
921614994014,22cyclictest926035-21cut20:35:010
92162499396,7cyclictest37-21ksoftirqd/300:20:083
92162499393,4cyclictest131rcu_preempt00:10:013
92162499393,3cyclictest131rcu_preempt21:22:103
921624993929,8cyclictest37-21ksoftirqd/321:42:373
921624993923,10cyclictest924665-21munin-plugin-st20:10:013
921624993917,14cyclictest37-21ksoftirqd/320:25:003
92162499391,6cyclictest0-21swapper/322:12:273
921619993933,4cyclictest923598-21kworker/2:120:42:452
92161999392,5cyclictest922566-21sshd19:27:052
921619993924,12cyclictest1-21systemd20:25:012
921614993928,8cyclictest12-21ksoftirqd/019:25:010
92162499385,3cyclictest131rcu_preempt21:10:003
92162499383,3cyclictest131rcu_preempt19:30:013
921624993827,7cyclictest37-21ksoftirqd/319:20:023
92162499382,4cyclictest131rcu_preempt22:22:013
92162499382,3cyclictest131rcu_preempt22:50:203
92162499381,3cyclictest131rcu_preempt00:37:173
92161499382,4cyclictest3467602-21systemd-journal19:48:060
92161499382,4cyclictest3467602-21systemd-journal19:48:060
92162499372,3cyclictest131rcu_preempt19:40:013
92161999375,27cyclictest926887-21grep20:50:022
92161999375,27cyclictest926887-21grep20:50:022
92161999372,4cyclictest131rcu_preempt19:25:012
92161999371,32cyclictest48-21kcompactd020:10:142
921614993729,5cyclictest16950irq/224-800000019:40:050
921614993727,7cyclictest12-21ksoftirqd/020:51:460
921614993725,9cyclictest12-21ksoftirqd/020:05:020
92161499372,31cyclictest3467718-21lldpd20:22:390
921624993622,10cyclictest924122-21cron20:00:003
921624993619,13cyclictest3467602-21systemd-journal19:15:003
92161499362,5cyclictest926727-21sshd20:48:050
92161499362,5cyclictest926727-21sshd20:48:040
92162499354,3cyclictest131rcu_preempt20:15:013
92162499353,3cyclictest131rcu_preempt20:29:053
921624993524,9cyclictest37-21ksoftirqd/320:50:013
921624993524,9cyclictest37-21ksoftirqd/320:50:013
921614993528,4cyclictest0-21swapper/019:54:370
92162499341,3cyclictest131rcu_preempt20:41:093
92161499341,30cyclictest0-21swapper/020:42:310
92162499314,3cyclictest131rcu_preempt20:55:063
921624992921,5cyclictest921741-21sed19:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional