You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-02 - 16:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot3s.osadl.org (updated Wed Sep 02, 2026 12:45:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2390801991001,4cyclictest0-21swapper/007:50:020
239080199952,4cyclictest131rcu_preempt10:10:020
2390801998926,13cyclictest2486776-21ssh12:19:370
2390801998914,14cyclictest275-21dbus-daemon09:10:000
2390801998623,12cyclictest2443497-21ssh10:47:340
2390801998617,64cyclictest16850irq/134-800000010:33:190
2390801998331,48cyclictest2432088-21ssh10:23:520
2390801998328,50cyclictest2484191-21ssh12:14:010
239080199831,37cyclictest2478868-21ssh12:02:360
2390801998125,4cyclictest16950irq/224-800000011:40:010
2390801998025,12cyclictest2426547-21ssh10:11:550
2390801998021,13cyclictest2405497-21ssh09:27:430
2390801998016,6cyclictest16850irq/134-800000010:42:490
2390801997927,16cyclictest2402990-21ssh09:22:260
2390801997917,12cyclictest2439585-21ssh10:39:380
2390801997867,9cyclictest19350irq/34-mmc111:40:020
239080199781,61cyclictest2450574-21ssh11:02:340
2390801997716,50cyclictest16850irq/134-800000011:48:020
2390801997618,14cyclictest2401081-21ssh09:18:290
2390801997611,56cyclictest16850irq/134-800000009:14:420
2390801997526,17cyclictest2458534-21ssh11:19:500
2390801997526,17cyclictest2458534-21ssh11:19:490
239080199751,6cyclictest2422660-21sshd10:04:000
239080199751,2cyclictest0-21swapper/009:58:420
239080199751,2cyclictest0-21swapper/009:30:540
239080199751,2cyclictest0-21swapper/009:30:540
2390801997512,57cyclictest16850irq/134-800000011:10:370
2390801997418,51cyclictest16850irq/134-800000011:57:560
239080199735,58cyclictest242922650ssh10:17:440
239080199733,6cyclictest2394878-21sshd08:25:290
239080199731,27cyclictest0-21swapper/011:52:040
2390801997217,45cyclictest16850irq/134-800000010:51:000
239080199711,65cyclictest0-21swapper/012:10:000
239081399702,17cyclictest2451921-21ssh11:05:063
239080199703,57cyclictest241574250ssh09:49:280
2390801997031,35cyclictest16950irq/224-800000012:29:590
239080199702,62cyclictest2461108-21ls11:25:020
2390801997021,16cyclictest2448125-21ssh10:57:190
2390801997021,16cyclictest2448125-21ssh10:57:190
2390813996923,41cyclictest3467562-21sshd09:54:293
2390813996923,41cyclictest3467562-21sshd09:54:293
239080199694,17cyclictest16850irq/134-800000011:08:550
2390801996915,6cyclictest16850irq/134-800000011:29:450
2390801996915,6cyclictest16850irq/134-800000011:29:450
2390813996828,36cyclictest2435077-21latency_hist10:30:023
2390813996823,40cyclictest2404146-21latency_hist09:25:023
239080199685,16cyclictest243467250ssh10:29:140
2390801996815,6cyclictest16850irq/134-800000012:37:370
2390801996717,15cyclictest2488346-21ssh12:22:370
2390801996716,46cyclictest16850irq/134-800000009:41:310
2390801996716,46cyclictest16850irq/134-800000009:41:310
2390813996621,39cyclictest0-21swapper/307:40:023
2390813996612,6cyclictest131rcu_preempt11:00:013
2390813996612,6cyclictest131rcu_preempt11:00:003
2390808996656,5cyclictest2390810-21latency_hist07:10:012
2390801996625,2cyclictest0-21swapper/009:52:210
2390801996625,2cyclictest0-21swapper/009:52:200
2390801996615,12cyclictest2492206-21ssh12:30:330
2390801996613,48cyclictest16850irq/134-800000011:34:240
2390813996524,37cyclictest37-21ksoftirqd/311:11:223
2390813996524,36cyclictest2394267-21latency_hist08:15:023
2390813996522,38cyclictest2396983-21diskmemload10:10:433
239081399652,17cyclictest2458646-21ssh11:20:013
239081399652,17cyclictest2458646-21ssh11:20:013
2390813996410,13cyclictest275-21dbus-daemon10:05:003
2390804996416,34cyclictest221rcuc/111:10:021
2390801996425,4cyclictest0-21swapper/008:04:280
239080199641,59cyclictest0-21swapper/007:20:010
2390813996333,26cyclictest3467602-21systemd-journal08:39:293
2390813996324,33cyclictest275-21dbus-daemon07:15:013
2390813996319,13cyclictest37-21ksoftirqd/309:15:563
2390813996317,11cyclictest2455818-1kworker/u9:111:32:233
2390808996242,16cyclictest3467602-21systemd-journal10:55:012
2390808996234,22cyclictest275-21dbus-daemon08:00:012
2390808996234,22cyclictest275-21dbus-daemon08:00:002
239081399612,18cyclictest2425486-21ssh10:10:003
2390813996120,10cyclictest37-21ksoftirqd/309:45:113
239080199611,27cyclictest0-21swapper/008:20:130
239080199611,27cyclictest0-21swapper/008:20:120
2390813996026,30cyclictest2411405-21cat09:40:033
2390813996024,32cyclictest2482174-21sh12:09:553
2390813996023,33cyclictest2437361-21latency_hist10:35:023
2390813996022,34cyclictest2399469-21rm09:15:013
2390808996051,6cyclictest2381569-21kworker/2:011:33:352
2390808996051,5cyclictest2381569-21kworker/2:010:56:172
2390808996051,5cyclictest2381569-21kworker/2:010:56:162
2390808996050,6cyclictest2381569-21kworker/2:009:43:192
2390808996050,6cyclictest2381569-21kworker/2:009:43:182
2390808996040,16cyclictest1-21systemd08:55:022
239080199601,26cyclictest0-21swapper/007:31:530
2390813995922,6cyclictest2432425-21sh10:24:333
2390813995919,10cyclictest37-21ksoftirqd/311:03:403
2390813995918,6cyclictest309-21in:imuxsock11:45:003
2390813995913,11cyclictest37-21ksoftirqd/309:44:003
2390813995913,11cyclictest37-21ksoftirqd/309:44:003
2390808995933,20cyclictest275-21dbus-daemon09:00:002
239080899592,43cyclictest2451756-21ssh11:05:012
2390813995820,8cyclictest37-21ksoftirqd/310:16:383
2390808995848,7cyclictest2381569-21kworker/2:010:48:312
2390801995824,3cyclictest0-21swapper/009:36:390
2390813995721,5cyclictest2462518-21sh11:28:003
2390813995721,5cyclictest2462518-21sh11:27:593
2390813995719,4cyclictest2396983-21diskmemload10:42:193
2390813995715,11cyclictest275-21dbus-daemon07:55:003
2390808995747,6cyclictest2395974-21kworker/2:109:22:212
2390808995741,6cyclictest2381569-21kworker/2:010:12:552
2390808995730,21cyclictest275-21dbus-daemon07:15:022
2390813995623,30cyclictest37-21ksoftirqd/309:25:313
2390813995623,29cyclictest2473941-21sh11:52:053
239081399562,18cyclictest2485268-21ssh12:16:063
2390813995621,10cyclictest37-21ksoftirqd/312:05:003
2390813995615,6cyclictest2458933-21sh11:20:093
239080899567,31cyclictest2391260-21modprobe07:20:002
2390808995648,5cyclictest2395974-21kworker/2:109:27:032
2390808995648,5cyclictest2381569-21kworker/2:011:16:462
2390808995648,5cyclictest2381569-21kworker/2:011:16:462
2390808995644,7cyclictest2381569-21kworker/2:012:10:152
2390808995632,17cyclictest275-21dbus-daemon08:10:012
2390801995620,32cyclictest2392424-21latency_hist07:40:030
2390813995514,7cyclictest37-21ksoftirqd/312:11:403
2390808995543,7cyclictest2381569-21kworker/2:010:34:422
239080499559,39cyclictest23-21ksoftirqd/108:20:011
2390801995520,7cyclictest12-21ksoftirqd/007:50:040
239080199551,28cyclictest0-21swapper/008:50:010
2390813995418,7cyclictest37-21ksoftirqd/309:30:353
2390813995418,7cyclictest37-21ksoftirqd/309:30:343
2390813995414,8cyclictest37-21ksoftirqd/312:25:033
2390808995445,6cyclictest2381569-21kworker/2:011:52:492
2390808995445,6cyclictest2381569-21kworker/2:010:37:572
2390808995445,5cyclictest2381569-21kworker/2:011:25:392
2390808995445,5cyclictest2381569-21kworker/2:011:25:382
2390808995445,5cyclictest2381569-21kworker/2:011:25:012
2390808995444,6cyclictest2381569-21kworker/2:010:16:502
2390808995433,15cyclictest275-21dbus-daemon08:45:002
2390813995329,21cyclictest2394386-21sendmail08:20:013
2390813995324,25cyclictest37-21ksoftirqd/310:53:363
2390813995318,9cyclictest2472828-21systemctl11:50:013
2390813995316,9cyclictest2430860-1kworker/u9:110:36:193
2390813995313,11cyclictest2474365-1kworker/u9:111:58:253
2390808995345,5cyclictest2381569-21kworker/2:011:48:462
2390808995345,5cyclictest2381569-21kworker/2:011:37:242
2390808995345,5cyclictest2381569-21kworker/2:010:05:362
2390808995345,5cyclictest2381569-21kworker/2:009:55:052
2390808995345,5cyclictest2381569-21kworker/2:008:18:392
2390808995344,6cyclictest2395974-21kworker/2:109:37:282
2390808995344,5cyclictest2461044-21kworker/2:112:35:202
2390808995343,7cyclictest2381569-21kworker/2:010:27:502
2390808995343,5cyclictest2381569-21kworker/2:010:04:372
2390801995346,3cyclictest0-21swapper/008:15:030
2390813995219,5cyclictest2443679-21rm10:48:083
2390813995219,10cyclictest2493544-21sh12:33:413
2390808995244,5cyclictest2381569-21kworker/2:011:09:572
2390808995244,5cyclictest2381569-21kworker/2:009:53:472
2390808995244,5cyclictest2381569-21kworker/2:009:53:462
2390808995244,5cyclictest2381569-21kworker/2:009:47:062
2390808995243,5cyclictest2461044-21kworker/2:112:31:532
2390808995243,5cyclictest2395974-21kworker/2:109:32:032
2390808995243,5cyclictest2395974-21kworker/2:109:32:022
2390808995242,7cyclictest2439840-21kworker/2:110:40:342
2390808995236,12cyclictest1-21systemd08:15:022
2390808995236,10cyclictest30-21ksoftirqd/207:45:002
2390808995230,15cyclictest275-21dbus-daemon08:50:002
2390801995213,13cyclictest1-21systemd07:30:000
2390813995123,25cyclictest37-21ksoftirqd/307:50:023
2390813995122,25cyclictest2420586-21sh09:59:393
239081399512,17cyclictest2495837-21ssh12:38:323
2390808995142,6cyclictest2461044-21kworker/2:111:42:452
2390808995142,6cyclictest2395974-21kworker/2:109:05:022
2390808995142,6cyclictest2395974-21kworker/2:109:05:012
2390808995142,6cyclictest2381569-21kworker/2:011:58:502
2390808995142,5cyclictest2395974-21kworker/2:109:10:012
2390801995137,10cyclictest275-21dbus-daemon08:35:000
2390813995019,10cyclictest2395083-21date08:30:023
239081399501,18cyclictest2488061-21ssh12:21:543
2390808995042,5cyclictest2395974-21kworker/2:109:19:592
2390808995042,5cyclictest2381569-21kworker/2:010:21:382
2390808995041,6cyclictest2461044-21kworker/2:112:19:192
2390808995041,6cyclictest2381569-21kworker/2:011:10:082
2390808995041,5cyclictest2461044-21kworker/2:112:20:172
2390804995035,12cyclictest1-21systemd12:40:011
2390801995020,8cyclictest12-21ksoftirqd/007:10:020
239080199501,28cyclictest0-21swapper/007:57:540
239080199501,28cyclictest0-21swapper/007:57:540
2390813994911,8cyclictest2392240-21date07:35:023
2390808994941,5cyclictest2461044-21kworker/2:112:27:262
2390808994941,5cyclictest2381569-21kworker/2:012:06:132
2390808994940,6cyclictest2395974-21kworker/2:109:14:422
2390808994940,5cyclictest2381569-21kworker/2:012:00:462
2390808994934,11cyclictest1-21systemd07:40:012
2390808994933,11cyclictest3467602-21systemd-journal08:40:022
2390804994932,12cyclictest3467602-21systemd-journal10:15:011
2390801994942,4cyclictest16950irq/224-800000009:00:010
239080199491,32cyclictest0-21swapper/008:11:290
239081399482,14cyclictest2396047-21cat08:50:013
2390813994817,4cyclictest2394658-21latency_hist08:25:023
2390813994817,4cyclictest2394658-21latency_hist08:25:023
2390813994810,12cyclictest2468088-21systemd-run11:40:013
2390808994840,5cyclictest2381569-21kworker/2:008:30:292
2390808994840,5cyclictest2381569-21kworker/2:008:05:022
2390804994837,8cyclictest1-21systemd11:55:001
2390804994832,12cyclictest23-21ksoftirqd/110:40:001
2390804994832,11cyclictest1-21systemd07:55:001
2390813994712,8cyclictest37-21ksoftirqd/308:00:023
2390813994712,8cyclictest37-21ksoftirqd/308:00:023
2390804994732,11cyclictest23-21ksoftirqd/112:00:001
2390804994632,11cyclictest23-21ksoftirqd/109:39:591
2390804994610,6cyclictest131rcu_preempt09:50:421
2390804994610,6cyclictest131rcu_preempt09:50:411
2390801994615,13cyclictest364-21snmpd07:13:300
2390813994519,5cyclictest37-21ksoftirqd/308:35:013
2390808994536,6cyclictest2381569-21kworker/2:008:28:462
2390804994530,12cyclictest275-21dbus-daemon11:35:011
239080499451,3cyclictest131rcu_preempt12:19:591
2390804994512,6cyclictest131rcu_preempt11:20:011
2390804994512,6cyclictest131rcu_preempt11:20:011
239080199451,27cyclictest0-21swapper/008:39:490
239081399448,6cyclictest131rcu_preempt09:00:013
2390808994436,5cyclictest2381569-21kworker/2:007:25:032
2390808994436,5cyclictest2381569-21kworker/2:007:25:022
2390804994432,9cyclictest23-21ksoftirqd/107:59:301
2390804994432,9cyclictest23-21ksoftirqd/107:59:291
2390804994427,12cyclictest275-21dbus-daemon11:00:011
2390804994427,12cyclictest275-21dbus-daemon11:00:001
239080499441,3cyclictest131rcu_preempt12:30:111
239080499441,3cyclictest131rcu_preempt10:23:221
239080199449,5cyclictest131rcu_preempt07:25:010
239080199449,5cyclictest131rcu_preempt07:25:000
2390801994435,7cyclictest12-21ksoftirqd/008:55:030
2390808994328,8cyclictest310-21in:imklog08:25:012
2390808994328,8cyclictest310-21in:imklog08:25:002
2390804994330,10cyclictest3467602-21systemd-journal10:55:011
2390804994328,11cyclictest23-21ksoftirqd/108:50:001
2390801994328,12cyclictest12-21ksoftirqd/008:43:450
2390808994232,6cyclictest2381569-21kworker/2:007:35:002
239080499425,8cyclictest23-21ksoftirqd/110:33:531
2390804994230,8cyclictest23-21ksoftirqd/112:05:001
239080499421,3cyclictest131rcu_preempt10:16:141
239081399412,4cyclictest131rcu_preempt08:45:003
2390808994133,5cyclictest2381569-21kworker/2:007:51:022
239080499413,3cyclictest131rcu_preempt11:24:501
2390804994128,9cyclictest2401785-21systemd-run09:20:011
2390804994124,12cyclictest275-21dbus-daemon12:10:011
239080499411,3cyclictest131rcu_preempt10:26:121
239080499411,3cyclictest131rcu_preempt10:09:051
2390801994124,13cyclictest275-21dbus-daemon08:10:000
2390808994031,6cyclictest2381569-21kworker/2:007:25:292
2390808994030,6cyclictest2381569-21kworker/2:007:50:012
2390804994027,10cyclictest3467602-21systemd-journal08:33:321
2390804994025,12cyclictest2399415-21systemd-run09:15:011
239080499401,4cyclictest131rcu_preempt11:44:531
239080499401,35cyclictest23-21ksoftirqd/111:46:481
239080499401,12cyclictest150-21pr/ttyS209:44:271
239080499401,12cyclictest150-21pr/ttyS209:44:261
239081399399,7cyclictest131rcu_preempt08:10:023
239081399391,3cyclictest0-21swapper/309:05:033
239081399391,3cyclictest0-21swapper/309:05:023
2390804993930,6cyclictest23-21ksoftirqd/108:05:011
2390804993929,8cyclictest23-21ksoftirqd/107:15:021
2390804993926,10cyclictest2404158-21systemd-run09:25:011
2390804993926,10cyclictest23-21ksoftirqd/112:29:321
2390804993924,11cyclictest23-21ksoftirqd/112:20:061
239080499391,3cyclictest131rcu_preempt11:37:411
2390804993911,5cyclictest131rcu_preempt08:55:021
239080199395,31cyclictest2392628-21date07:45:010
2390813993827,7cyclictest2391341-21cat07:20:023
2390813993826,9cyclictest1-21systemd07:25:013
2390813993826,9cyclictest1-21systemd07:25:013
239080499383,4cyclictest131rcu_preempt11:03:411
2390804993829,7cyclictest23-21ksoftirqd/107:40:011
2390804993828,7cyclictest2395418-21sendmail08:40:001
2390804993828,7cyclictest23-21ksoftirqd/107:30:011
2390804993827,9cyclictest23-21ksoftirqd/112:13:161
2390804993826,10cyclictest23-21ksoftirqd/109:45:521
2390804993825,9cyclictest3467602-21systemd-journal09:33:561
2390804993825,9cyclictest3467602-21systemd-journal09:33:561
2390804993824,10cyclictest275-21dbus-daemon10:05:001
2390804993823,11cyclictest23-21ksoftirqd/109:10:011
2390804993822,12cyclictest1-21systemd11:10:011
239080499382,11cyclictest2463462-21cat11:30:021
239080499382,11cyclictest2463462-21cat11:30:011
2390813993722,11cyclictest1-21systemd07:30:013
239080499373,5cyclictest131rcu_preempt08:10:011
2390804993728,7cyclictest23-21ksoftirqd/109:26:561
2390804993716,14cyclictest275-21dbus-daemon07:35:001
239080499371,3cyclictest131rcu_preempt10:45:261
239081399364,5cyclictest131rcu_preempt08:55:023
239080499367,3cyclictest131rcu_preempt07:25:021
239080499367,3cyclictest131rcu_preempt07:25:011
2390804993628,6cyclictest23-21ksoftirqd/108:45:021
2390804993625,8cyclictest2394643-21sshd08:23:411
2390804993625,8cyclictest2394643-21sshd08:23:401
2390804993624,9cyclictest23-21ksoftirqd/110:41:211
2390813993524,6cyclictest2396989-21sh09:09:593
239080499353,3cyclictest131rcu_preempt07:50:021
2390804993526,5cyclictest2396477-21lockfile-create09:00:011
2390804993524,9cyclictest23-21ksoftirqd/109:55:571
2390804993523,9cyclictest23-21ksoftirqd/107:43:181
2390804993521,11cyclictest23-21ksoftirqd/108:13:471
2390804993521,10cyclictest23-21ksoftirqd/108:30:021
2390813993417,12cyclictest275-21dbus-daemon08:05:013
2390804993421,10cyclictest23-21ksoftirqd/107:20:001
239080199341,29cyclictest0-21swapper/009:01:020
239080199341,29cyclictest0-21swapper/009:01:010
2390813993321,8cyclictest2390952-21cut07:10:033
2390804993324,6cyclictest23-21ksoftirqd/109:05:001
2390804993324,6cyclictest23-21ksoftirqd/109:05:001
2390804993220,8cyclictest0-21swapper/107:10:031
2390813993123,5cyclictest2392757-21latency_hist07:45:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional